Update man pages

This reverts commit ef090d425e.

COPYING

@@ -1,6 +1,6 @@
 The MIT License
 
-Copyright (c) 2012, 2014 Tatsuhiro Tsujikawa
+Copyright (c) 2012, 2014, 2015 Tatsuhiro Tsujikawa
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

README.rst

@@ -110,25 +110,9 @@ The Python bindings require the following packages:
 * python >= 2.7
 
 If you are using Ubuntu 14.04 LTS, you need the following packages
-installed:
+installed::
 
-* make
-* binutils
-* autoconf
-* automake
-* autotools-dev
-* libtool
-* pkg-config
-* zlib1g-dev
-* libcunit1-dev
-* libssl-dev
-* libxml2-dev
-* libev-dev
-* libevent-dev
-* libjansson-dev
-* libjemalloc-dev
-* cython
-* python3.4-dev
+    apt-get install make binutils autoconf  automake autotools-dev libtool pkg-config zlib1g-dev libcunit1-dev libssl-dev libxml2-dev libev-dev libevent-dev libjansson-dev libjemalloc-dev cython python3.4-dev
 
 spdylay is not packaged in Ubuntu, so you need to build it yourself:
 http://tatsuhiro-t.github.io/spdylay/
@@ -214,7 +198,7 @@ with prior knowledge, HTTP Upgrade and NPN/ALPN TLS extension.
 It has verbose output mode for framing information.  Here is sample
 output from ``nghttp`` client::
 
-    $ src/nghttp -nv https://nghttp2.org
+    $ nghttp -nv https://nghttp2.org
     [  0.033][NPN] server offers:
               * h2-14
               * spdy/3.1
@@ -283,7 +267,7 @@ output from ``nghttp`` client::
 
 The HTTP Upgrade is performed like this::
 
-    $ src/nghttp -nvu http://nghttp2.org
+    $ nghttp -nvu http://nghttp2.org
     [  0.013] HTTP Upgrade request
     GET / HTTP/1.1
     Host: nghttp2.org
@@ -350,6 +334,35 @@ The HTTP Upgrade is performed like this::
     [  0.038] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
               (last_stream_id=0, error_code=NO_ERROR(0), opaque_data(0)=[])
 
+With ``-s`` option, ``nghttp`` prints out some timing information for
+requests, sorted by completion time::
+
+    $ nghttp -nas https://nghttp2.org/
+    ***** Statistics *****
+
+    Request timing:
+      complete: relative time from protocol handshake to stream close
+       request: relative   time  from   protocol   handshake  to   request
+                transmission.  If '*' is shown, this was pushed by server.
+       process: time for request and response
+          code: HTTP status code
+          size: number  of   bytes  received  as  response   body  without
+                inflation.
+           URI: request URI
+
+    sorted by 'complete'
+
+    complete  request    process  code size request path
+     +11.07ms     +120us  10.95ms  200   9K /
+     +16.77ms *  +8.80ms   7.98ms  200   8K /stylesheets/screen.css
+     +27.00ms   +11.16ms  15.84ms  200   3K /javascripts/octopress.js
+     +27.40ms   +11.16ms  16.24ms  200   3K /javascripts/modernizr-2.0.js
+     +76.14ms   +11.17ms  64.97ms  200 171K /images/posts/with-pri-blog.png
+     +88.52ms   +11.17ms  77.36ms  200 174K /images/posts/without-pri-blog.png
+
+With ``-r`` option, ``nghttp`` writes more detailed timing data to
+given file in HAR format.
+
 nghttpd - server
 ++++++++++++++++
 
@@ -366,7 +379,7 @@ HTTP/2 connection.  No HTTP Upgrade is supported.
 Just like ``nghttp``, it has verbose output mode for framing
 information.  Here is sample output from ``nghttpd`` server::
 
-    $ src/nghttpd --no-tls -v 8080
+    $ nghttpd --no-tls -v 8080
     IPv4: listen on port 8080
     IPv6: listen on port 8080
     [id=1] [ 15.921] send SETTINGS frame <length=10, flags=0x00, stream_id=0>
@@ -418,7 +431,8 @@ nghttpx - proxy
 +++++++++++++++
 
 ``nghttpx`` is a multi-threaded reverse proxy for ``h2-14``, SPDY and
-HTTP/1.1 and powers nghttp2.org site.  It has several operation modes:
+HTTP/1.1 and powers nghttp2.org site and supports HTTP/2 server push.
+It has several operation modes:
 
 ================== ============================ ============== =============
 Mode option        Frontend                     Backend        Note
@@ -530,8 +544,11 @@ library.  The UI of ``h2load`` is heavily inspired by ``weighttp``
 (https://github.com/lighttpd/weighttp).  The typical usage is as
 follows::
 
-    $ src/h2load -n1000 -c10 -m10 https://127.0.0.1:8443/
+    $ h2load -n100000 -c100 -m100 https://localhost:8443/
     starting benchmark...
+    spawning thread #0: 100 concurrent clients, 100000 total requests
+    Protocol: TLSv1.2
+    Cipher: ECDHE-RSA-AES128-GCM-SHA256
     progress: 10% done
     progress: 20% done
     progress: 30% done
@@ -543,15 +560,17 @@ follows::
     progress: 90% done
     progress: 100% done
 
-    finished in 0 sec, 152 millisec and 152 microsec, 6572 req/s, 749 kbytes/s
-    requests: 1000 total, 1000 started, 1000 done, 0 succeeded, 1000 failed, 0 errored
-    status codes: 0 2xx, 0 3xx, 1000 4xx, 0 5xx
-    traffic: 141100 bytes total, 840 bytes headers, 116000 bytes data
+    finished in 7.10s, 14092 req/s, 55.67MB/s
+    requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
+    status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
+    traffic: 414200800 bytes total, 2723100 bytes headers, 409600000 bytes data
+                         min         max         mean         sd        +/- sd
+    time for request:   283.86ms       1.46s    659.70ms    150.87ms    84.68%
 
-The above example issued total 1000 requests, using 10 concurrent
-clients (thus 10 HTTP/2 sessions), and maximum 10 streams per client.
-With ``-t`` option, ``h2load`` will use multiple native threads to
-avoid saturating single core on client side.
+The above example issued total 100000 requests, using 100 concurrent
+clients (in other words, 100 HTTP/2 sessions), and maximum 100 streams
+per client.  With ``-t`` option, ``h2load`` will use multiple native
+threads to avoid saturating single core on client side.
 
 .. warning::
 

configure.ac

@@ -25,14 +25,14 @@ dnl Do not change user variables!
 dnl http://www.gnu.org/software/automake/manual/html_node/Flag-Variables-Ordering.html
 
 AC_PREREQ(2.61)
-AC_INIT([nghttp2], [0.7.2], [t-tujikawa@users.sourceforge.net])
+AC_INIT([nghttp2], [0.7.5], [t-tujikawa@users.sourceforge.net])
 LT_PREREQ([2.2.6])
 LT_INIT()
 dnl See versioning rule:
 dnl  http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
-AC_SUBST(LT_CURRENT, 9)
+AC_SUBST(LT_CURRENT, 11)
 AC_SUBST(LT_REVISION, 0)
-AC_SUBST(LT_AGE, 4)
+AC_SUBST(LT_AGE, 6)
 
 major=`echo $PACKAGE_VERSION |cut -d. -f1 | sed -e "s/[^0-9]//g"`
 minor=`echo $PACKAGE_VERSION |cut -d. -f2 | sed -e "s/[^0-9]//g"`

doc/Makefile.am

@@ -41,24 +41,28 @@ EXTRA_DIST = \
 	sources/python-apiref.rst \
 	sources/building-android-binary.rst \
 	sources/contribute.rst \
-	_themes/sphinx_rtd_theme/footer.html \
-	_themes/sphinx_rtd_theme/theme.conf \
-	_themes/sphinx_rtd_theme/layout_old.html \
 	_themes/sphinx_rtd_theme/__init__.py \
-	_themes/sphinx_rtd_theme/layout.html \
-	_themes/sphinx_rtd_theme/search.html \
 	_themes/sphinx_rtd_theme/breadcrumbs.html \
-	_themes/sphinx_rtd_theme/versions.html \
+	_themes/sphinx_rtd_theme/footer.html \
+	_themes/sphinx_rtd_theme/layout.html \
+	_themes/sphinx_rtd_theme/layout_old.html \
+	_themes/sphinx_rtd_theme/search.html \
 	_themes/sphinx_rtd_theme/searchbox.html \
-	_themes/sphinx_rtd_theme/static/fonts/FontAwesome.otf \
-	_themes/sphinx_rtd_theme/static/fonts/fontawesome-webfont.svg \
-	_themes/sphinx_rtd_theme/static/fonts/fontawesome-webfont.woff \
-	_themes/sphinx_rtd_theme/static/fonts/fontawesome-webfont.eot \
-	_themes/sphinx_rtd_theme/static/fonts/fontawesome-webfont.ttf \
-	_themes/sphinx_rtd_theme/static/js/theme.js \
-	_themes/sphinx_rtd_theme/static/css/theme.css \
 	_themes/sphinx_rtd_theme/static/css/badge_only.css \
-	$(man_MANS)
+	_themes/sphinx_rtd_theme/static/css/theme.css \
+	_themes/sphinx_rtd_theme/static/fonts/FontAwesome.otf \
+	_themes/sphinx_rtd_theme/static/fonts/fontawesome-webfont.eot \
+	_themes/sphinx_rtd_theme/static/fonts/fontawesome-webfont.svg \
+	_themes/sphinx_rtd_theme/static/fonts/fontawesome-webfont.ttf \
+	_themes/sphinx_rtd_theme/static/fonts/fontawesome-webfont.woff \
+	_themes/sphinx_rtd_theme/static/js/theme.js \
+	_themes/sphinx_rtd_theme/theme.conf \
+	_themes/sphinx_rtd_theme/versions.html \
+	$(man_MANS) \
+	bash_completion/nghttp \
+	bash_completion/nghttpd \
+	bash_completion/nghttpx \
+	bash_completion/h2load
 
 # Makefile for Sphinx documentation
 #
@@ -104,7 +108,7 @@ clean-local:
 	-rm apiref.rst
 	-rm -rf $(BUILDDIR)/*
 
-html: apiref.rst
+html-local: apiref.rst
 	$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
 	@echo
 	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

doc/apiref-header.rst

@@ -31,3 +31,70 @@ do not consume client connection preface unless
 value.  The applications are responsible to receive it before calling
 these functions if :type:`nghttp2_session` is configured as server and
 `nghttp2_option_set_recv_client_preface()` is not used.
+
+HTTP Messaging
+--------------
+
+By default, nghttp2 library checks HTTP messaging rules described in
+`HTTP/2 specification, section 8
+<https://tools.ietf.org/html/draft-ietf-httpbis-http2-17#section-8>`_.
+Everything described in that section is not validated however.  We
+briefly describe what the library does in this area.  In the following
+description, without loss of generality we omit CONTINUATION frame
+since they must follow HEADERS frame and are processed atomically.  In
+other words, they are just one big HEADERS frame.  To disable these
+validations, use `nghttp2_option_set_no_http_messaging()`.
+
+For HTTP request, including those carried by PUSH_PROMISE, HTTP
+message starts with one HEADERS frame containing request headers.  It
+is followed by zero or more DATA frames containing request body, which
+is followed by zero or one HEADERS containing trailer headers.  The
+request headers must include ":scheme", ":method" and ":path" pseudo
+header fields unless ":method" is not "CONNECT".  ":authority" is
+optional, but nghttp2 requires either ":authority" or "Host" header
+field must be present.  If ":method" is "CONNECT", the request headers
+must include ":method" and ":authority" and must omit ":scheme" and
+":path".
+
+For HTTP response, HTTP message starts with zero or more HEADERS
+frames containing non-final response (status code 1xx).  They are
+followed by one HEADERS frame containing final response headers
+(non-1xx).  It is followed by zero or more DATA frames containing
+response body, which is followed by zero or one HEADERS containing
+trailer headers.  The non-final and final response headers must
+contain ":status" pseudo header field containing 3 digits only.
+
+All request and response headers must include exactly one valid value
+for each pseudo header field.  Additionally nghttp2 requires all
+request headers must not include more than one "Host" header field.
+
+HTTP/2 prohibits connection-specific header fields.  The following
+header fields must not appear: "Connection", "Keep-Alive",
+"Proxy-Connection", "Transfer-Encoding" and "Upgrade".  Additionally,
+"TE" header field must not include any value other than "trailers".
+
+Each header field name and value must obey the field-name and
+field-value production rules described in `RFC 7230, section
+3.2. <https://tools.ietf.org/html/rfc7230#section-3.2>`_.
+Additionally, all field name must be lower cased.  While the pseudo
+header fields must satisfy these rules, we just ignore illegal regular
+headers (this means that these header fields are not passed to
+application callback).  This is because these illegal header fields
+are floating around in existing internet and resetting stream just
+because of this may break many web sites.  This is especially true if
+we forward to or translate from HTTP/1 traffic.
+
+With the above validations, nghttp2 library guarantees that header
+field name passed to `nghttp2_on_header_callback()` is not empty.
+Also required pseudo headers are all present and not empty.
+
+nghttp2 enforces "Content-Length" validation as well.  All request or
+response headers must not contain more than one "Content-Length"
+header field.  If "Content-Length" header field is present, it must be
+parsed as 64 bit signed integer.  The sum of data length in the
+following DATA frames must match with the number in "Content-Length"
+header field if it is present (this does not include padding bytes).
+
+Any deviation results in stream error of type PROTOCOL_ERROR.  If
+error is found in PUSH_PROMISE frame, stream error is raised against
+promised stream.

doc/bash_completion/h2load

@@ -0,0 +1,19 @@
+_h2load()
+{
+    local cur prev split=false
+    COMPREPLY=()
+    COMP_WORDBREAKS=${COMP_WORDBREAKS//=}
+
+    cmd=${COMP_WORDS[0]}
+    _get_comp_words_by_ref cur prev
+    case $cur in
+        -*)
+            COMPREPLY=( $( compgen -W '--threads --connection-window-bits --input-file --help --requests --verbose --version --window-bits --clients --no-tls-proto --header --max-concurrent-streams ' -- "$cur" ) )
+            ;;
+        *)
+            _filedir
+            return 0
+    esac
+    return 0
+}
+complete -F _h2load h2load

doc/bash_completion/make_bash_completion.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python
+
+import subprocess
+from StringIO import StringIO
+import re
+import sys
+import os.path
+
+class Option:
+    def __init__(self, long_opt, short_opt):
+        self.long_opt = long_opt
+        self.short_opt = short_opt
+
+def get_all_options(cmd):
+    opt_pattern = re.compile(r'  (?:(-.), )?(--[^\s\[=]+)(\[)?')
+    proc = subprocess.Popen([cmd, "--help"], stdout=subprocess.PIPE)
+    stdoutdata, stderrdata = proc.communicate()
+    cur_option = None
+    opts = {}
+    for line in StringIO(stdoutdata):
+        match = opt_pattern.match(line)
+        if not match:
+            continue
+        long_opt = match.group(2)
+        short_opt = match.group(1)
+        opts[long_opt] = Option(long_opt, short_opt)
+
+    return opts
+
+def output_case(out, name, opts):
+    out.write('''\
+_{name}()
+{{
+    local cur prev split=false
+    COMPREPLY=()
+    COMP_WORDBREAKS=${{COMP_WORDBREAKS//=}}
+
+    cmd=${{COMP_WORDS[0]}}
+    _get_comp_words_by_ref cur prev
+'''.format(name=name))
+
+    # Complete option name.
+    out.write('''\
+    case $cur in
+        -*)
+            COMPREPLY=( $( compgen -W '\
+''')
+    for opt in opts.itervalues():
+        out.write(opt.long_opt)
+        out.write(' ')
+
+    out.write('''\
+' -- "$cur" ) )
+            ;;
+''')
+    # If no option found for completion then complete with files.
+    out.write('''\
+        *)
+            _filedir
+            return 0
+    esac
+    return 0
+}}
+complete -F _{name} {name}
+'''.format(name=name))
+
+if __name__ == '__main__':
+    if len(sys.argv) < 2:
+        print "Generates bash_completion using `/path/to/cmd --help'"
+        print "Usage: make_bash_completion.py /path/to/cmd"
+        exit(1)
+    name = os.path.basename(sys.argv[1])
+    opts = get_all_options(sys.argv[1])
+    output_case(sys.stdout, name, opts)

doc/bash_completion/nghttp

@@ -0,0 +1,19 @@
+_nghttp()
+{
+    local cur prev split=false
+    COMPREPLY=()
+    COMP_WORDBREAKS=${COMP_WORDBREAKS//=}
+
+    cmd=${COMP_WORDS[0]}
+    _get_comp_words_by_ref cur prev
+    case $cur in
+        -*)
+            COMPREPLY=( $( compgen -W '--verbose --no-dep --get-assets --har --header-table-size --multiply --padding --dep-idle --continuation --connection-window-bits --peer-max-concurrent-streams --timeout --data --no-content-length --version --color --cert --upgrade --remote-name --weight --help --key --null-out --window-bits --stat --header ' -- "$cur" ) )
+            ;;
+        *)
+            _filedir
+            return 0
+    esac
+    return 0
+}
+complete -F _nghttp nghttp

doc/bash_completion/nghttpd

@@ -0,0 +1,19 @@
+_nghttpd()
+{
+    local cur prev split=false
+    COMPREPLY=()
+    COMP_WORDBREAKS=${COMP_WORDBREAKS//=}
+
+    cmd=${COMP_WORDS[0]}
+    _get_comp_words_by_ref cur prev
+    case $cur in
+        -*)
+            COMPREPLY=( $( compgen -W '--error-gzip --push --header-table-size --htdocs --padding --verbose --version --help --daemon --verify-client --workers --no-tls --color --early-response --dh-param-file ' -- "$cur" ) )
+            ;;
+        *)
+            _filedir
+            return 0
+    esac
+    return 0
+}
+complete -F _nghttpd nghttpd

doc/bash_completion/nghttpx

@@ -0,0 +1,19 @@
+_nghttpx()
+{
+    local cur prev split=false
+    COMPREPLY=()
+    COMP_WORDBREAKS=${COMP_WORDBREAKS//=}
+
+    cmd=${COMP_WORDS[0]}
+    _get_comp_words_by_ref cur prev
+    case $cur in
+        -*)
+            COMPREPLY=( $( compgen -W '--frontend-http2-connection-window-bits --worker-read-rate --frontend-no-tls --frontend-http2-dump-request-header --daemon --write-rate --altsvc --frontend-http2-dump-response-header --backend-http1-connections-per-frontend --tls-ticket-key-file --ciphers --verify-client-cacert --backend-keep-alive-timeout --strip-incoming-x-forwarded-for --errorlog-file --private-key-passwd-file --version --backlog --backend-http-proxy-uri --add-response-header --backend-write-timeout --backend-request-buffer --add-x-forwarded-for --write-burst --backend-http2-connection-window-bits --insecure --rlimit-nofile --backend-http2-window-bits --tls-proto-list --no-location-rewrite --padding --accesslog-syslog --conf --http2-max-concurrent-streams --client-proxy --worker-frontend-connections --cacert --frontend-read-timeout --worker-write-burst --npn-list --syslog-facility --backend-http1-connections-per-host --no-server-push --client --http2-bridge --no-via --user --stream-write-timeout --backend-response-buffer --http2-no-cookie-crumbling --backend-read-timeout --stream-read-timeout --workers --worker-read-burst --tls-ctx-per-worker --dh-param-file --errorlog-syslog --frontend --accesslog-file --http2-proxy --read-burst --accesslog-format --frontend-http2-window-bits --backend-no-tls --client-private-key-file --pid-file --client-cert-file --no-host-rewrite --log-level --worker-write-rate --help --backend-tls-sni-field --subcert --frontend-frame-debug --frontend-write-timeout --verify-client --read-rate --frontend-http2-read-timeout --backend-ipv4 --listener-disable-timeout --backend-ipv6 --backend ' -- "$cur" ) )
+            ;;
+        *)
+            _filedir
+            return 0
+    esac
+    return 0
+}
+complete -F _nghttpx nghttpx

doc/h2load.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "H2LOAD" "1" "January 25, 2015" "0.7.2" "nghttp2"
+.TH "H2LOAD" "1" "February 27, 2015" "0.7.5" "nghttp2"
 .SH NAME
 h2load \- HTTP/2 benchmarking tool
 .
@@ -46,7 +46,7 @@ so  on.  The  scheme, host  and port  in the  subsequent
 URIs, if present,  are ignored.  Those in  the first URI
 are used solely.
 .UNINDENT
-.SH OPTIONS:
+.SH OPTIONS
 .INDENT 0.0
 .TP
 .B \-n, \-\-requests=<N>
@@ -131,6 +131,77 @@ Display version information and exit.
 .B \-h, \-\-help
 Display this help and exit.
 .UNINDENT
+.SH OUTPUT
+.INDENT 0.0
+.TP
+.B requests
+.INDENT 7.0
+.TP
+.B total
+The number of requests h2load was instructed to make.
+.TP
+.B started
+The number of requests h2load has started.
+.TP
+.B done
+The number of requests completed.
+.TP
+.B succeeded
+The number of requests completed successfully.  Only HTTP status
+code 2xx or3xx are considered as success.
+.TP
+.B failed
+The number of requests failed, including HTTP level failures
+(non\-successful HTTP status code).
+.TP
+.B errored
+The number of requests failed, except for HTTP level failures.
+status code.  This is the subset of the number reported in
+\fBfailed\fP and most likely the network level failures or stream
+was reset by RST_STREAM.
+.UNINDENT
+.TP
+.B status codes
+The number of status code h2load received.
+.TP
+.B traffic
+.INDENT 7.0
+.TP
+.B total
+The number of bytes received from the server "on the wire".  If
+requests were made via TLS, this value is the number of decrpyted
+bytes.
+.TP
+.B headers
+The number of response header bytes from the server without
+decompression.  For HTTP/2, this is the sum of the payload of
+HEADERS frame.  For SPDY, this is the sum of the payload of
+SYN_REPLY frame.
+.TP
+.B data
+The number of response body bytes received from the server.
+.UNINDENT
+.TP
+.B time for request
+.INDENT 7.0
+.TP
+.B min
+The minimum time taken for request and response.
+.TP
+.B max
+The maximum time taken for request and response.
+.TP
+.B mean
+The mean time taken for request and response.
+.TP
+.B sd
+The standard deviation of the time for request and response.
+.TP
+.B +/\- sd
+The fraction of the number of requests within standard deviation
+range (mean +/\- sd) against total number of successful requests.
+.UNINDENT
+.UNINDENT
 .SH SEE ALSO
 .sp
 \fInghttp(1)\fP, \fInghttpd(1)\fP, \fInghttpx(1)\fP

doc/h2load.1.rst

@@ -21,8 +21,8 @@ benchmarking tool for HTTP/2 and SPDY server
     URIs, if present,  are ignored.  Those in  the first URI
     are used solely.
 
-OPTIONS:
---------
+OPTIONS
+-------
 
 .. option:: -n, --requests=<N>
 
@@ -96,6 +96,57 @@ OPTIONS:
 
     Display this help and exit.
 
+OUTPUT
+------
+
+requests
+  total
+    The number of requests h2load was instructed to make.
+  started
+    The number of requests h2load has started.
+  done
+    The number of requests completed.
+  succeeded
+    The number of requests completed successfully.  Only HTTP status
+    code 2xx or3xx are considered as success.
+  failed
+    The number of requests failed, including HTTP level failures
+    (non-successful HTTP status code).
+  errored
+    The number of requests failed, except for HTTP level failures.
+    status code.  This is the subset of the number reported in
+    ``failed`` and most likely the network level failures or stream
+    was reset by RST_STREAM.
+
+status codes
+  The number of status code h2load received.
+
+traffic
+  total
+    The number of bytes received from the server "on the wire".  If
+    requests were made via TLS, this value is the number of decrpyted
+    bytes.
+  headers
+    The number of response header bytes from the server without
+    decompression.  For HTTP/2, this is the sum of the payload of
+    HEADERS frame.  For SPDY, this is the sum of the payload of
+    SYN_REPLY frame.
+  data
+    The number of response body bytes received from the server.
+
+time for request
+  min
+    The minimum time taken for request and response.
+  max
+    The maximum time taken for request and response.
+  mean
+    The mean time taken for request and response.
+  sd
+    The standard deviation of the time for request and response.
+  +/- sd
+    The fraction of the number of requests within standard deviation
+    range (mean +/- sd) against total number of successful requests.
+
 SEE ALSO
 --------
 

doc/h2load.h2r

@@ -1,3 +1,54 @@
+OUTPUT
+------
+
+requests
+  total
+    The number of requests h2load was instructed to make.
+  started
+    The number of requests h2load has started.
+  done
+    The number of requests completed.
+  succeeded
+    The number of requests completed successfully.  Only HTTP status
+    code 2xx or3xx are considered as success.
+  failed
+    The number of requests failed, including HTTP level failures
+    (non-successful HTTP status code).
+  errored
+    The number of requests failed, except for HTTP level failures.
+    status code.  This is the subset of the number reported in
+    ``failed`` and most likely the network level failures or stream
+    was reset by RST_STREAM.
+
+status codes
+  The number of status code h2load received.
+
+traffic
+  total
+    The number of bytes received from the server "on the wire".  If
+    requests were made via TLS, this value is the number of decrpyted
+    bytes.
+  headers
+    The number of response header bytes from the server without
+    decompression.  For HTTP/2, this is the sum of the payload of
+    HEADERS frame.  For SPDY, this is the sum of the payload of
+    SYN_REPLY frame.
+  data
+    The number of response body bytes received from the server.
+
+time for request
+  min
+    The minimum time taken for request and response.
+  max
+    The maximum time taken for request and response.
+  mean
+    The mean time taken for request and response.
+  sd
+    The standard deviation of the time for request and response.
+  +/- sd
+    The fraction of the number of requests within standard deviation
+    range (mean +/- sd) against total number of successful requests.
+
 SEE ALSO
 --------
 

doc/nghttp.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTP" "1" "January 25, 2015" "0.7.2" "nghttp2"
+.TH "NGHTTP" "1" "February 27, 2015" "0.7.5" "nghttp2"
 .SH NAME
 nghttp \- HTTP/2 experimental client
 .
@@ -41,7 +41,7 @@ HTTP/2 experimental client
 .B <URI>
 Specify URI to access.
 .UNINDENT
-.SH OPTIONS:
+.SH OPTIONS
 .INDENT 0.0
 .TP
 .B \-v, \-\-verbose

doc/nghttp.1.rst

@@ -16,8 +16,8 @@ HTTP/2 experimental client
 
     Specify URI to access.
 
-OPTIONS:
---------
+OPTIONS
+-------
 
 .. option:: -v, --verbose
 

doc/nghttpd.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTPD" "1" "January 25, 2015" "0.7.2" "nghttp2"
+.TH "NGHTTPD" "1" "February 27, 2015" "0.7.5" "nghttp2"
 .SH NAME
 nghttpd \- HTTP/2 experimental server
 .
@@ -53,7 +53,13 @@ Set  path  to  server\(aqs private  key.   Required  unless
 Set  path  to  server\(aqs  certificate.   Required  unless
 \fI\%\-\-no\-tls\fP is specified.
 .UNINDENT
-.SH OPTIONS:
+.SH OPTIONS
+.INDENT 0.0
+.TP
+.B \-a, \-\-address=<ADDR>
+The address to bind to.  If not specified the default IP
+address determined by getaddrinfo is used.
+.UNINDENT
 .INDENT 0.0
 .TP
 .B \-D, \-\-daemon

doc/nghttpd.1.rst

@@ -27,8 +27,13 @@ HTTP/2 experimental server
     Set  path  to  server's  certificate.   Required  unless
     :option:`--no-tls` is specified.
 
-OPTIONS:
---------
+OPTIONS
+-------
+
+.. option:: -a, --address=<ADDR>
+
+    The address to bind to.  If not specified the default IP
+    address determined by getaddrinfo is used.
 
 .. option:: -D, --daemon
 

doc/nghttpx.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTPX" "1" "January 25, 2015" "0.7.2" "nghttp2"
+.TH "NGHTTPX" "1" "February 27, 2015" "0.7.5" "nghttp2"
 .SH NAME
 nghttpx \- HTTP/2 experimental proxy
 .
@@ -48,10 +48,10 @@ Set path  to server\(aqs private key.   Required unless \fI\%\-p\fP,
 Set path  to server\(aqs certificate.  Required  unless \fI\%\-p\fP,
 \fI\%\-\-client\fP or \fI\%\-\-frontend\-no\-tls\fP are given.
 .UNINDENT
-.SH OPTIONS:
+.SH OPTIONS
 .sp
 The options are categorized into several groups.
-.SS Connections:
+.SS Connections
 .INDENT 0.0
 .TP
 .B \-b, \-\-backend=<HOST,PORT>
@@ -59,7 +59,9 @@ Set backend host and port.  For HTTP/1 backend, multiple
 backend addresses are accepted by repeating this option.
 HTTP/2  backend   does  not  support   multiple  backend
 addresses  and the  first occurrence  of this  option is
-used.
+used.  UNIX domain socket  can be specified by prefixing
+path        name        with       "unix:"        (e.g.,
+unix:/var/run/backend.sock)
 .sp
 Default: \fB127.0.0.1,80\fP
 .UNINDENT
@@ -67,7 +69,9 @@ Default: \fB127.0.0.1,80\fP
 .TP
 .B \-f, \-\-frontend=<HOST,PORT>
 Set  frontend  host and  port.   If  <HOST> is  \(aq*\(aq,  it
-assumes all addresses including both IPv4 and IPv6.
+assumes  all addresses  including  both  IPv4 and  IPv6.
+UNIX domain  socket can  be specified by  prefixing path
+name with "unix:" (e.g., unix:/var/run/nghttpx.sock)
 .sp
 Default: \fB*,3000\fP
 .UNINDENT
@@ -104,7 +108,7 @@ timeouts when connecting and  making CONNECT request can
 be     specified    by     \fI\%\-\-backend\-read\-timeout\fP    and
 \fI\%\-\-backend\-write\-timeout\fP options.
 .UNINDENT
-.SS Performance:
+.SS Performance
 .INDENT 0.0
 .TP
 .B \-n, \-\-workers=<N>
@@ -234,32 +238,32 @@ Set buffer size used to store backend response.
 .sp
 Default: \fB16K\fP
 .UNINDENT
-.SS Timeout:
+.SS Timeout
 .INDENT 0.0
 .TP
-.B \-\-frontend\-http2\-read\-timeout=<SEC>
+.B \-\-frontend\-http2\-read\-timeout=<DURATION>
 Specify  read  timeout  for  HTTP/2  and  SPDY  frontend
 connection.
 .sp
-Default: \fB180\fP
+Default: \fB180s\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-frontend\-read\-timeout=<SEC>
+.B \-\-frontend\-read\-timeout=<DURATION>
 Specify read timeout for HTTP/1.1 frontend connection.
 .sp
-Default: \fB180\fP
+Default: \fB180s\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-frontend\-write\-timeout=<SEC>
+.B \-\-frontend\-write\-timeout=<DURATION>
 Specify write timeout for all frontend connections.
 .sp
-Default: \fB30\fP
+Default: \fB30s\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-stream\-read\-timeout=<SEC>
+.B \-\-stream\-read\-timeout=<DURATION>
 Specify  read timeout  for HTTP/2  and SPDY  streams.  0
 means no timeout.
 .sp
@@ -267,7 +271,7 @@ Default: \fB0\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-stream\-write\-timeout=<SEC>
+.B \-\-stream\-write\-timeout=<DURATION>
 Specify write  timeout for  HTTP/2 and SPDY  streams.  0
 means no timeout.
 .sp
@@ -275,35 +279,35 @@ Default: \fB0\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-backend\-read\-timeout=<SEC>
+.B \-\-backend\-read\-timeout=<DURATION>
 Specify read timeout for backend connection.
 .sp
-Default: \fB180\fP
+Default: \fB180s\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-backend\-write\-timeout=<SEC>
+.B \-\-backend\-write\-timeout=<DURATION>
 Specify write timeout for backend connection.
 .sp
-Default: \fB30\fP
+Default: \fB30s\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-backend\-keep\-alive\-timeout=<SEC>
+.B \-\-backend\-keep\-alive\-timeout=<DURATION>
 Specify keep\-alive timeout for backend connection.
 .sp
-Default: \fB600\fP
+Default: \fB2s\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-listener\-disable\-timeout=<SEC>
+.B \-\-listener\-disable\-timeout=<DURATION>
 After accepting  connection failed,  connection listener
-is disabled for  a given time in  seconds.  Specifying 0
+is disabled  for a given  amount of time.   Specifying 0
 disables this feature.
 .sp
 Default: \fB0\fP
 .UNINDENT
-.SS SSL/TLS:
+.SS SSL/TLS
 .INDENT 0.0
 .TP
 .B \-\-ciphers=<SUITE>
@@ -365,7 +369,7 @@ NPN.  The parameter must be  delimited by a single comma
 only  and any  white spaces  are  treated as  a part  of
 protocol string.
 .sp
-Default: \fBh2\-16,h2\-14,spdy/3.1,http/1.1\fP
+Default: \fBh2,h2\-16,h2\-14,spdy/3.1,http/1.1\fP
 .UNINDENT
 .INDENT 0.0
 .TP
@@ -434,7 +438,7 @@ objects, which means session  ID generated by one worker
 is not acceptable by another worker.  On the other hand,
 session ticket key is shared across all worker threads.
 .UNINDENT
-.SS HTTP/2 and SPDY:
+.SS HTTP/2 and SPDY
 .INDENT 0.0
 .TP
 .B \-c, \-\-http2\-max\-concurrent\-streams=<N>
@@ -500,7 +504,14 @@ padding.  Specify 0 to  disable padding.  This option is
 meant for debugging purpose  and not intended to enhance
 protocol security.
 .UNINDENT
-.SS Mode:
+.INDENT 0.0
+.TP
+.B \-\-no\-server\-push
+Disable  HTTP/2  server  push.    Server  push  is  only
+supported  by default  mode and  HTTP/2 frontend.   SPDY
+frontend does not support server push.
+.UNINDENT
+.SS Mode
 .INDENT 0.0
 .TP
 .B (default mode)
@@ -541,7 +552,7 @@ Like \fI\%\-\-client\fP  option, but it also  requires the request
 path from frontend must be an absolute URI, suitable for
 use as a forward proxy.
 .UNINDENT
-.SS Logging:
+.SS Logging
 .INDENT 0.0
 .TP
 .B \-L, \-\-log\-level=<LEVEL>
@@ -623,7 +634,7 @@ Set syslog facility to <FACILITY>.
 .sp
 Default: \fBdaemon\fP
 .UNINDENT
-.SS HTTP:
+.SS HTTP
 .INDENT 0.0
 .TP
 .B \-\-add\-x\-forwarded\-for
@@ -652,6 +663,14 @@ altered regardless of this option.
 .UNINDENT
 .INDENT 0.0
 .TP
+.B \-\-no\-host\-rewrite
+Don\(aqt  rewrite  host  and :authority  header  fields  on
+\fI\%\-\-http2\-bridge\fP,   \fI\%\-\-client\fP   and  default   mode.    For
+\fI\%\-\-http2\-proxy\fP  and  \fI\%\-\-client\-proxy\fP mode,  these  headers
+will not be altered regardless of this option.
+.UNINDENT
+.INDENT 0.0
+.TP
 .B \-\-altsvc=<PROTOID,PORT[,HOST,[ORIGIN]]>
 Specify   protocol  ID,   port,  host   and  origin   of
 alternative service.  <HOST>  and <ORIGIN> are optional.
@@ -669,7 +688,7 @@ won\(aqt replace anything already  set.  This option can be
 used several  times to  specify multiple  header fields.
 Example: \fI\%\-\-add\-response\-header\fP="foo: bar"
 .UNINDENT
-.SS Debug:
+.SS Debug
 .INDENT 0.0
 .TP
 .B \-\-frontend\-http2\-dump\-request\-header=<PATH>
@@ -695,7 +714,7 @@ Print HTTP/2 frames in  frontend to stderr.  This option
 is  not thread  safe and  MUST NOT  be used  with option
 \fI\%\-n\fP=N, where N >= 2.
 .UNINDENT
-.SS Process:
+.SS Process
 .INDENT 0.0
 .TP
 .B \-D, \-\-daemon
@@ -713,7 +732,7 @@ Set path to save PID of this program.
 Run this program as <USER>.   This option is intended to
 be used to drop root privileges.
 .UNINDENT
-.SS Misc:
+.SS Misc
 .INDENT 0.0
 .TP
 .B \-\-conf=<PATH>
@@ -734,6 +753,10 @@ Print this help and exit.
 .sp
 The <SIZE> argument is an integer and an optional unit (e.g., 10K is
 10 * 1024).  Units are K, M and G (powers of 1024).
+.sp
+The <DURATION> argument is an integer and an optional unit (e.g., 1s
+is 1 second and 500ms is 500  milliseconds).  Units are s or ms.  If
+a unit is omitted, a second is used as unit.
 .SH FILES
 .INDENT 0.0
 .TP
@@ -778,6 +801,50 @@ path with same command\-line arguments and environment variables.
 After new process comes up, sending SIGQUIT to the original process
 to perform hot swapping.
 .UNINDENT
+.SH SERVER PUSH
+.sp
+nghttpx supports HTTP/2 server push in default mode.  nghttpx looks
+for Link header field (\fI\%RFC 5988\fP) in response headers from
+backend server and extracts URI\-reference with parameter
+\fBrel=preload\fP (see \fI\%preload\fP)
+and pushes those URIs to the frontend client. Here is a sample Link
+header field to initiate server push:
+.INDENT 0.0
+.INDENT 3.5
+.sp
+.nf
+.ft C
+Link: </fonts/font.woff>; rel=preload
+Link: </css/theme.css>; rel=preload
+.ft P
+.fi
+.UNINDENT
+.UNINDENT
+.sp
+Currently, the following restrictions are applied for server push:
+.INDENT 0.0
+.IP 1. 3
+URI\-reference must not contain authority.  If it exists, it is not
+pushed.  \fB/fonts/font.woff\fP and \fBcss/theme.css\fP are eligible to
+be pushed.  \fBhttps://example.org/fonts/font.woff\fP and
+\fB//example.org/css/theme.css\fP are not.
+.IP 2. 3
+The associated stream must have method "GET" or "POST".  The
+associated stream\(aqs status code must be 200.
+.UNINDENT
+.sp
+These limitations may be loosened in the future release.
+.SH UNIX DOMAIN SOCKET
+.sp
+nghttpx supports UNIX domain socket with a filename for both frontend
+and backend connections.
+.sp
+Please note that current nghttpx implementation does not delete a
+socket with a filename.  And on start up, if nghttpx detects that the
+specified socket already exists in the file system, nghttpx first
+deletes it.  However, if SIGUSR2 is used to execute new binary and
+both old and new configurations use same filename, new binary does not
+delete the socket and continues to use it.
 .SH SEE ALSO
 .sp
 \fInghttp(1)\fP, \fInghttpd(1)\fP, \fIh2load(1)\fP

doc/nghttpx.1.rst

@@ -24,13 +24,13 @@ A reverse proxy for HTTP/2, HTTP/1 and SPDY.
     :option:`--client` or :option:`\--frontend-no-tls` are given.
 
 
-OPTIONS:
---------
+OPTIONS
+-------
 
 The options are categorized into several groups.
 
-Connections:
-~~~~~~~~~~~~
+Connections
+~~~~~~~~~~~
 
 .. option:: -b, --backend=<HOST,PORT>
 
@@ -38,14 +38,18 @@ Connections:
     backend addresses are accepted by repeating this option.
     HTTP/2  backend   does  not  support   multiple  backend
     addresses  and the  first occurrence  of this  option is
-    used.
+    used.  UNIX domain socket  can be specified by prefixing
+    path        name        with       "unix:"        (e.g.,
+    unix:/var/run/backend.sock)
 
     Default: ``127.0.0.1,80``
 
 .. option:: -f, --frontend=<HOST,PORT>
 
     Set  frontend  host and  port.   If  <HOST> is  '\*',  it
-    assumes all addresses including both IPv4 and IPv6.
+    assumes  all addresses  including  both  IPv4 and  IPv6.
+    UNIX domain  socket can  be specified by  prefixing path
+    name with "unix:" (e.g., unix:/var/run/nghttpx.sock)
 
     Default: ``*,3000``
 
@@ -79,8 +83,8 @@ Connections:
     :option:`--backend-write-timeout` options.
 
 
-Performance:
-~~~~~~~~~~~~
+Performance
+~~~~~~~~~~~
 
 .. option:: -n, --workers=<N>
 
@@ -197,71 +201,71 @@ Performance:
     Default: ``16K``
 
 
-Timeout:
-~~~~~~~~
+Timeout
+~~~~~~~
 
-.. option:: --frontend-http2-read-timeout=<SEC>
+.. option:: --frontend-http2-read-timeout=<DURATION>
 
     Specify  read  timeout  for  HTTP/2  and  SPDY  frontend
     connection.
 
-    Default: ``180``
+    Default: ``180s``
 
-.. option:: --frontend-read-timeout=<SEC>
+.. option:: --frontend-read-timeout=<DURATION>
 
     Specify read timeout for HTTP/1.1 frontend connection.
 
-    Default: ``180``
+    Default: ``180s``
 
-.. option:: --frontend-write-timeout=<SEC>
+.. option:: --frontend-write-timeout=<DURATION>
 
     Specify write timeout for all frontend connections.
 
-    Default: ``30``
+    Default: ``30s``
 
-.. option:: --stream-read-timeout=<SEC>
+.. option:: --stream-read-timeout=<DURATION>
 
     Specify  read timeout  for HTTP/2  and SPDY  streams.  0
     means no timeout.
 
     Default: ``0``
 
-.. option:: --stream-write-timeout=<SEC>
+.. option:: --stream-write-timeout=<DURATION>
 
     Specify write  timeout for  HTTP/2 and SPDY  streams.  0
     means no timeout.
 
     Default: ``0``
 
-.. option:: --backend-read-timeout=<SEC>
+.. option:: --backend-read-timeout=<DURATION>
 
     Specify read timeout for backend connection.
 
-    Default: ``180``
+    Default: ``180s``
 
-.. option:: --backend-write-timeout=<SEC>
+.. option:: --backend-write-timeout=<DURATION>
 
     Specify write timeout for backend connection.
 
-    Default: ``30``
+    Default: ``30s``
 
-.. option:: --backend-keep-alive-timeout=<SEC>
+.. option:: --backend-keep-alive-timeout=<DURATION>
 
     Specify keep-alive timeout for backend connection.
 
-    Default: ``600``
+    Default: ``2s``
 
-.. option:: --listener-disable-timeout=<SEC>
+.. option:: --listener-disable-timeout=<DURATION>
 
     After accepting  connection failed,  connection listener
-    is disabled for  a given time in  seconds.  Specifying 0
+    is disabled  for a given  amount of time.   Specifying 0
     disables this feature.
 
     Default: ``0``
 
 
-SSL/TLS:
-~~~~~~~~
+SSL/TLS
+~~~~~~~
 
 .. option:: --ciphers=<SUITE>
 
@@ -316,7 +320,7 @@ SSL/TLS:
     only  and any  white spaces  are  treated as  a part  of
     protocol string.
 
-    Default: ``h2-16,h2-14,spdy/3.1,http/1.1``
+    Default: ``h2,h2-16,h2-14,spdy/3.1,http/1.1``
 
 .. option:: --verify-client
 
@@ -379,8 +383,8 @@ SSL/TLS:
     session ticket key is shared across all worker threads.
 
 
-HTTP/2 and SPDY:
-~~~~~~~~~~~~~~~~
+HTTP/2 and SPDY
+~~~~~~~~~~~~~~~
 
 .. option:: -c, --http2-max-concurrent-streams=<N>
 
@@ -438,9 +442,15 @@ HTTP/2 and SPDY:
     meant for debugging purpose  and not intended to enhance
     protocol security.
 
+.. option:: --no-server-push
 
-Mode:
-~~~~~
+    Disable  HTTP/2  server  push.    Server  push  is  only
+    supported  by default  mode and  HTTP/2 frontend.   SPDY
+    frontend does not support server push.
+
+
+Mode
+~~~~
 
 .. describe:: (default mode)
 
@@ -479,8 +489,8 @@ Mode:
     use as a forward proxy.
 
 
-Logging:
-~~~~~~~~
+Logging
+~~~~~~~
 
 .. option:: -L, --log-level=<LEVEL>
 
@@ -545,8 +555,8 @@ Logging:
     Default: ``daemon``
 
 
-HTTP:
-~~~~~
+HTTP
+~~~~
 
 .. option:: --add-x-forwarded-for
 
@@ -570,6 +580,13 @@ HTTP:
     :option:`--client-proxy` mode,  location header field will  not be
     altered regardless of this option.
 
+.. option:: --no-host-rewrite
+
+    Don't  rewrite  host  and :authority  header  fields  on
+    :option:`--http2-bridge`\,   :option:`--client`   and  default   mode.    For
+    :option:`--http2-proxy`  and  :option:`\--client-proxy` mode,  these  headers
+    will not be altered regardless of this option.
+
 .. option:: --altsvc=<PROTOID,PORT[,HOST,[ORIGIN]]>
 
     Specify   protocol  ID,   port,  host   and  origin   of
@@ -588,8 +605,8 @@ HTTP:
     Example: :option:`--add-response-header`\="foo: bar"
 
 
-Debug:
-~~~~~~
+Debug
+~~~~~
 
 .. option:: --frontend-http2-dump-request-header=<PATH>
 
@@ -614,8 +631,8 @@ Debug:
     :option:`-n`\=N, where N >= 2.
 
 
-Process:
-~~~~~~~~
+Process
+~~~~~~~
 
 .. option:: -D, --daemon
 
@@ -632,8 +649,8 @@ Process:
     be used to drop root privileges.
 
 
-Misc:
-~~~~~
+Misc
+~~~~
 
 .. option:: --conf=<PATH>
 
@@ -653,6 +670,10 @@ Misc:
 The <SIZE> argument is an integer and an optional unit (e.g., 10K is
 10 * 1024).  Units are K, M and G (powers of 1024).
 
+The <DURATION> argument is an integer and an optional unit (e.g., 1s
+is 1 second and 500ms is 500  milliseconds).  Units are s or ms.  If
+a unit is omitted, a second is used as unit.
+
 FILES
 -----
 
@@ -697,6 +718,48 @@ SIGUSR2
   After new process comes up, sending SIGQUIT to the original process
   to perform hot swapping.
 
+SERVER PUSH
+-----------
+
+nghttpx supports HTTP/2 server push in default mode.  nghttpx looks
+for Link header field (`RFC 5988
+<http://tools.ietf.org/html/rfc5988>`_) in response headers from
+backend server and extracts URI-reference with parameter
+``rel=preload`` (see `preload
+<http://w3c.github.io/preload/#interoperability-with-http-link-header>`_)
+and pushes those URIs to the frontend client. Here is a sample Link
+header field to initiate server push:
+
+.. code-block:: http
+
+  Link: </fonts/font.woff>; rel=preload
+  Link: </css/theme.css>; rel=preload
+
+Currently, the following restrictions are applied for server push:
+
+1. URI-reference must not contain authority.  If it exists, it is not
+   pushed.  ``/fonts/font.woff`` and ``css/theme.css`` are eligible to
+   be pushed.  ``https://example.org/fonts/font.woff`` and
+   ``//example.org/css/theme.css`` are not.
+
+2. The associated stream must have method "GET" or "POST".  The
+   associated stream's status code must be 200.
+
+These limitations may be loosened in the future release.
+
+UNIX DOMAIN SOCKET
+------------------
+
+nghttpx supports UNIX domain socket with a filename for both frontend
+and backend connections.
+
+Please note that current nghttpx implementation does not delete a
+socket with a filename.  And on start up, if nghttpx detects that the
+specified socket already exists in the file system, nghttpx first
+deletes it.  However, if SIGUSR2 is used to execute new binary and
+both old and new configurations use same filename, new binary does not
+delete the socket and continues to use it.
+
 SEE ALSO
 --------
 

doc/nghttpx.h2r

@@ -42,6 +42,48 @@ SIGUSR2
   After new process comes up, sending SIGQUIT to the original process
   to perform hot swapping.
 
+SERVER PUSH
+-----------
+
+nghttpx supports HTTP/2 server push in default mode.  nghttpx looks
+for Link header field (`RFC 5988
+<http://tools.ietf.org/html/rfc5988>`_) in response headers from
+backend server and extracts URI-reference with parameter
+``rel=preload`` (see `preload
+<http://w3c.github.io/preload/#interoperability-with-http-link-header>`_)
+and pushes those URIs to the frontend client. Here is a sample Link
+header field to initiate server push:
+
+.. code-block:: http
+
+  Link: </fonts/font.woff>; rel=preload
+  Link: </css/theme.css>; rel=preload
+
+Currently, the following restrictions are applied for server push:
+
+1. URI-reference must not contain authority.  If it exists, it is not
+   pushed.  ``/fonts/font.woff`` and ``css/theme.css`` are eligible to
+   be pushed.  ``https://example.org/fonts/font.woff`` and
+   ``//example.org/css/theme.css`` are not.
+
+2. The associated stream must have method "GET" or "POST".  The
+   associated stream's status code must be 200.
+
+These limitations may be loosened in the future release.
+
+UNIX DOMAIN SOCKET
+------------------
+
+nghttpx supports UNIX domain socket with a filename for both frontend
+and backend connections.
+
+Please note that current nghttpx implementation does not delete a
+socket with a filename.  And on start up, if nghttpx detects that the
+specified socket already exists in the file system, nghttpx first
+deletes it.  However, if SIGUSR2 is used to execute new binary and
+both old and new configurations use same filename, new binary does not
+delete the socket and continues to use it.
+
 SEE ALSO
 --------
 

doc/sources/nghttpx-howto.rst

@@ -198,8 +198,8 @@ backend connection, use ``--backend-no-tls`` option.
 
 The backend server is supporsed to be a HTTP/2 web server or HTTP/2
 proxy.  If backend server is HTTP/2 proxy, use
-``--no-location-rewrite`` option to disable rewriting location header
-field.
+``--no-location-rewrite`` and ``--no-host-rewrite`` options to disable
+rewriting location, host and :authority header field.
 
 The use-case of this mode is aggregate the incoming connections to one
 HTTP/2 connection.  One backend HTTP/2 connection is created per

examples/asio-sv2.cc

@@ -93,8 +93,7 @@ int main(int argc, char *argv[]) {
       if (stat(path.c_str(), &stbuf) == 0) {
         headers.push_back(
             header{"content-length", std::to_string(stbuf.st_size)});
-        headers.push_back(
-            header{"last-modified", http_date(stbuf.st_mtim.tv_sec)});
+        headers.push_back(header{"last-modified", http_date(stbuf.st_mtime)});
       }
       res->write_head(200, std::move(headers));
       res->end(file_reader_from_fd(fd));

examples/client.c

@@ -558,6 +558,8 @@ static void fetch_uri(const struct URI *uri) {
     diec("nghttp2_session_client_new", rv);
   }
 
+  nghttp2_submit_settings(connection.session, NGHTTP2_FLAG_NONE, NULL, 0);
+
   /* Submit the HTTP request to the outbound queue. */
   submit_request(&connection, &req);
 

examples/tiny-nghttpd.c

@@ -179,13 +179,57 @@ static char *io_buf_add_str(io_buf *buf, const void *src, size_t len) {
   return (char *)start;
 }
 
-static int memseq(const uint8_t *a, size_t alen, const char *b) {
-  const uint8_t *last = a + alen;
+static int memeq(const void *a, const void *b, size_t n) {
+  return memcmp(a, b, n) == 0;
+}
 
-  for (; a != last && *b && *a == *b; ++a, ++b)
-    ;
+#define streq(A, B, N) ((sizeof((A)) - 1) == (N) && memeq((A), (B), (N)))
 
-  return a == last && *b == 0;
+typedef enum {
+  NGHTTP2_TOKEN__AUTHORITY,
+  NGHTTP2_TOKEN__METHOD,
+  NGHTTP2_TOKEN__PATH,
+  NGHTTP2_TOKEN__SCHEME,
+  NGHTTP2_TOKEN_HOST,
+} nghttp2_token;
+
+/* Inspired by h2o header lookup.  https://github.com/h2o/h2o */
+static int lookup_token(const uint8_t *name, size_t namelen) {
+  switch (namelen) {
+  case 5:
+    switch (name[namelen - 1]) {
+    case 'h':
+      if (streq(":pat", name, 4)) {
+        return NGHTTP2_TOKEN__PATH;
+      }
+      break;
+    }
+    break;
+  case 7:
+    switch (name[namelen - 1]) {
+    case 'd':
+      if (streq(":metho", name, 6)) {
+        return NGHTTP2_TOKEN__METHOD;
+      }
+      break;
+    case 'e':
+      if (streq(":schem", name, 6)) {
+        return NGHTTP2_TOKEN__SCHEME;
+      }
+      break;
+    }
+    break;
+  case 10:
+    switch (name[namelen - 1]) {
+    case 'y':
+      if (streq(":authorit", name, 9)) {
+        return NGHTTP2_TOKEN__AUTHORITY;
+      }
+      break;
+    }
+    break;
+  }
+  return -1;
 }
 
 static char *cpydig(char *buf, int n, size_t len) {
@@ -925,8 +969,7 @@ static int on_header_callback(nghttp2_session *session,
                               const nghttp2_frame *frame, const uint8_t *name,
                               size_t namelen, const uint8_t *value,
                               size_t valuelen, uint8_t flags _U_,
-                              void *user_data) {
-  connection *conn = user_data;
+                              void *user_data _U_) {
   stream *strm;
 
   if (frame->hd.type != NGHTTP2_HEADERS ||
@@ -940,74 +983,42 @@ static int on_header_callback(nghttp2_session *session,
     return 0;
   }
 
-  if (!nghttp2_check_header_name(name, namelen) ||
-      !nghttp2_check_header_value(value, valuelen)) {
-    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-  }
-
-  if (memseq(name, namelen, ":method")) {
-    if (strm->method) {
-      stream_error(conn, strm->stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
+  switch (lookup_token(name, namelen)) {
+  case NGHTTP2_TOKEN__METHOD:
     strm->method = io_buf_add_str(&strm->scrbuf, value, valuelen);
     if (!strm->method) {
       return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
     }
     strm->methodlen = valuelen;
-    return 0;
-  }
-
-  if (memseq(name, namelen, ":scheme")) {
-    if (strm->scheme) {
-      stream_error(conn, strm->stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
+    break;
+  case NGHTTP2_TOKEN__SCHEME:
     strm->scheme = io_buf_add_str(&strm->scrbuf, value, valuelen);
     if (!strm->scheme) {
       return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
     }
     strm->schemelen = valuelen;
-    return 0;
-  }
-
-  if (memseq(name, namelen, ":authority")) {
-    if (strm->authority) {
-      stream_error(conn, strm->stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
+    break;
+  case NGHTTP2_TOKEN__AUTHORITY:
     strm->authority = io_buf_add_str(&strm->scrbuf, value, valuelen);
     if (!strm->authority) {
       return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
     }
     strm->authoritylen = valuelen;
-    return 0;
-  }
-
-  if (memseq(name, namelen, ":path")) {
-    if (strm->path) {
-      stream_error(conn, strm->stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
+    break;
+  case NGHTTP2_TOKEN__PATH:
     strm->path = io_buf_add_str(&strm->scrbuf, value, valuelen);
     if (!strm->path) {
       return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
     }
     strm->pathlen = valuelen;
-    return 0;
-  }
-
-  if (name[0] == ':') {
-    stream_error(conn, strm->stream_id, NGHTTP2_PROTOCOL_ERROR);
-    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-  }
-
-  if (memseq(name, namelen, "host") && !strm->host) {
+    break;
+  case NGHTTP2_TOKEN_HOST:
     strm->host = io_buf_add_str(&strm->scrbuf, value, valuelen);
     if (!strm->host) {
       return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
     }
     strm->hostlen = valuelen;
+    break;
   }
 
   return 0;
@@ -1029,12 +1040,6 @@ static int on_frame_recv_callback(nghttp2_session *session,
     return 0;
   }
 
-  if (!strm->method || !strm->scheme || !strm->path ||
-      (!strm->authority && !strm->host)) {
-    stream_error(conn, strm->stream_id, NGHTTP2_PROTOCOL_ERROR);
-    return 0;
-  }
-
   if (!strm->host) {
     strm->host = strm->authority;
     strm->hostlen = strm->authoritylen;

genheaderfunc.py

@@ -21,6 +21,11 @@ HEADERS = [
     "content-length",
     "location",
     "trailer",
+    "link",
+    "accept-encoding",
+    "accept-language",
+    "cache-control",
+    "user-agent",
     # disallowed h1 headers
     'connection',
     'keep-alive',
@@ -79,7 +84,7 @@ int lookup_token(const uint8_t *name, size_t namelen) {
     case '{}':'''.format(c)
             for k in headers:
                 print '''\
-      if (util::streq("{}", name, {})) {{
+      if (util::streq_l("{}", name, {})) {{
         return {};
       }}'''.format(k[:-1], size - 1, to_enum_hd(k))
             print '''\

genlibtokenlookup.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python
+
+HEADERS = [
+    ':authority',
+    ':method',
+    ':path',
+    ':scheme',
+    ':status',
+    "content-length",
+    "host",
+    "te",
+    'connection',
+    'keep-alive',
+    'proxy-connection',
+    'transfer-encoding',
+    'upgrade'
+]
+
+def to_enum_hd(k):
+    res = 'NGHTTP2_TOKEN_'
+    for c in k.upper():
+        if c == ':' or c == '-':
+            res += '_'
+            continue
+        res += c
+    return res
+
+def build_header(headers):
+    res = {}
+    for k in headers:
+        size = len(k)
+        if size not in res:
+            res[size] = {}
+        ent = res[size]
+        c = k[-1]
+        if c not in ent:
+            ent[c] = []
+        ent[c].append(k)
+
+    return res
+
+def gen_enum():
+    print '''\
+typedef enum {'''
+    for k in sorted(HEADERS):
+        print '''\
+  {},'''.format(to_enum_hd(k))
+    print '''\
+  NGHTTP2_TOKEN_MAXIDX,
+} nghttp2_token;'''
+
+def gen_index_header():
+    print '''\
+static int lookup_token(const uint8_t *name, size_t namelen) {
+  switch (namelen) {'''
+    b = build_header(HEADERS)
+    for size in sorted(b.keys()):
+        ents = b[size]
+        print '''\
+  case {}:'''.format(size)
+        print '''\
+    switch (name[namelen - 1]) {'''
+        for c in sorted(ents.keys()):
+            headers = sorted(ents[c])
+            print '''\
+    case '{}':'''.format(c)
+            for k in headers:
+                print '''\
+      if (streq("{}", name, {})) {{
+        return {};
+      }}'''.format(k[:-1], size - 1, to_enum_hd(k))
+            print '''\
+      break;'''
+        print '''\
+    }
+    break;'''
+    print '''\
+  }
+  return -1;
+}'''
+
+if __name__ == '__main__':
+    gen_enum()
+    print ''
+    gen_index_header()

help2rst.py

@@ -96,8 +96,8 @@ DESCRIPTION
             in_arg = False
 
         if line == 'Options:':
-            print 'OPTIONS:'
-            print '--------'
+            print 'OPTIONS'
+            print '-------'
             print ''
             continue
 
@@ -137,7 +137,7 @@ DESCRIPTION
 
         if not line.startswith(' ') and line.endswith(':'):
             # subsection
-            subsec = line.strip()
+            subsec = line.strip()[:-1]
             print '{}'.format(subsec)
             print '{}'.format('~' * len(subsec))
             print ''

integration-tests/nghttpx_http1_test.go

@@ -139,6 +139,78 @@ func TestH1H1GracefulShutdown(t *testing.T) {
 	}
 }
 
+// TestH1H1HostRewrite tests that server rewrites Host header field
+func TestH1H1HostRewrite(t *testing.T) {
+	st := newServerTester(nil, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name: "TestH1H1HostRewrite",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := res.header.Get("request-host"), st.backendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}
+
+// TestH1H1HTTP10 tests that server can accept HTTP/1.0 request
+// without Host header field
+func TestH1H1HTTP10(t *testing.T) {
+	st := newServerTester(nil, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	if _, err := io.WriteString(st.conn, "GET / HTTP/1.0\r\nTest-Case: TestH1H1HTTP10\r\n\r\n"); err != nil {
+		t.Fatalf("Error io.WriteString() = %v", err)
+	}
+
+	resp, err := http.ReadResponse(bufio.NewReader(st.conn), nil)
+	if err != nil {
+		t.Fatalf("Error http.ReadResponse() = %v", err)
+	}
+
+	if got, want := resp.StatusCode, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := resp.Header.Get("request-host"), st.backendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}
+
+// TestH1H1HTTP10NoHostRewrite tests that server generates host header
+// field using actual backend server even if --no-http-rewrite is
+// used.
+func TestH1H1HTTP10NoHostRewrite(t *testing.T) {
+	st := newServerTester([]string{"--no-host-rewrite"}, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	if _, err := io.WriteString(st.conn, "GET / HTTP/1.0\r\nTest-Case: TestH1H1HTTP10NoHostRewrite\r\n\r\n"); err != nil {
+		t.Fatalf("Error io.WriteString() = %v", err)
+	}
+
+	resp, err := http.ReadResponse(bufio.NewReader(st.conn), nil)
+	if err != nil {
+		t.Fatalf("Error http.ReadResponse() = %v", err)
+	}
+
+	if got, want := resp.StatusCode, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := resp.Header.Get("request-host"), st.backendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}
+
 // TestH1H2ConnectFailure tests that server handles the situation that
 // connection attempt to HTTP/2 backend failed.
 func TestH1H2ConnectFailure(t *testing.T) {
@@ -184,6 +256,57 @@ func TestH1H2NoHost(t *testing.T) {
 	}
 }
 
+// TestH1H2HTTP10 tests that server can accept HTTP/1.0 request
+// without Host header field
+func TestH1H2HTTP10(t *testing.T) {
+	st := newServerTester([]string{"--http2-bridge"}, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	if _, err := io.WriteString(st.conn, "GET / HTTP/1.0\r\nTest-Case: TestH1H2HTTP10\r\n\r\n"); err != nil {
+		t.Fatalf("Error io.WriteString() = %v", err)
+	}
+
+	resp, err := http.ReadResponse(bufio.NewReader(st.conn), nil)
+	if err != nil {
+		t.Fatalf("Error http.ReadResponse() = %v", err)
+	}
+
+	if got, want := resp.StatusCode, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := resp.Header.Get("request-host"), st.backendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}
+
+// TestH1H2HTTP10NoHostRewrite tests that server generates host header
+// field using actual backend server even if --no-http-rewrite is
+// used.
+func TestH1H2HTTP10NoHostRewrite(t *testing.T) {
+	st := newServerTester([]string{"--http2-bridge", "--no-host-rewrite"}, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	if _, err := io.WriteString(st.conn, "GET / HTTP/1.0\r\nTest-Case: TestH1H2HTTP10NoHostRewrite\r\n\r\n"); err != nil {
+		t.Fatalf("Error io.WriteString() = %v", err)
+	}
+
+	resp, err := http.ReadResponse(bufio.NewReader(st.conn), nil)
+	if err != nil {
+		t.Fatalf("Error http.ReadResponse() = %v", err)
+	}
+
+	if got, want := resp.StatusCode, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := resp.Header.Get("request-host"), st.backendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}
+
 // TestH1H2CrumbleCookie tests that Cookies are crumbled and assembled
 // when forwarding to HTTP/2 backend link.  go-nghttp2 server
 // concatenates crumbled Cookies automatically, so this test is not
@@ -209,3 +332,74 @@ func TestH1H2CrumbleCookie(t *testing.T) {
 		t.Errorf("status: %v; want %v", got, want)
 	}
 }
+
+// TestH1H2GenerateVia tests that server generates Via header field to and
+// from backend server.
+func TestH1H2GenerateVia(t *testing.T) {
+	st := newServerTester([]string{"--http2-bridge"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "1.1 nghttpx"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+	})
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name: "TestH1H2GenerateVia",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "2 nghttpx"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestH1H2AppendVia tests that server adds value to existing Via
+// header field to and from backend server.
+func TestH1H2AppendVia(t *testing.T) {
+	st := newServerTester([]string{"--http2-bridge"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "foo, 1.1 nghttpx"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+		w.Header().Add("Via", "bar")
+	})
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name: "TestH1H2AppendVia",
+		header: []hpack.HeaderField{
+			pair("via", "foo"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "bar, 2 nghttpx"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestH1H2NoVia tests that server does not add value to existing Via
+// header field to and from backend server.
+func TestH1H2NoVia(t *testing.T) {
+	st := newServerTester([]string{"--http2-bridge", "--no-via"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "foo"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+		w.Header().Add("Via", "bar")
+	})
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name: "TestH1H2NoVia",
+		header: []hpack.HeaderField{
+			pair("via", "foo"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "bar"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}

integration-tests/nghttpx_http2_test.go

@@ -12,7 +12,7 @@ import (
 	"testing"
 )
 
-// TestH1H2PlainGET tests whether simple HTTP/2 GET request works.
+// TestH2H1PlainGET tests whether simple HTTP/2 GET request works.
 func TestH2H1PlainGET(t *testing.T) {
 	st := newServerTester(nil, t, noopHandler)
 	defer st.Close()
@@ -121,6 +121,120 @@ func TestH2H1StripAddXff(t *testing.T) {
 	}
 }
 
+// TestH2H1GenerateVia tests that server generates Via header field to and
+// from backend server.
+func TestH2H1GenerateVia(t *testing.T) {
+	st := newServerTester(nil, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "2 nghttpx"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2H1GenerateVia",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "1.1 nghttpx"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestH2H1AppendVia tests that server adds value to existing Via
+// header field to and from backend server.
+func TestH2H1AppendVia(t *testing.T) {
+	st := newServerTester(nil, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "foo, 2 nghttpx"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+		w.Header().Add("Via", "bar")
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2H1AppendVia",
+		header: []hpack.HeaderField{
+			pair("via", "foo"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "bar, 1.1 nghttpx"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestH2H1NoVia tests that server does not add value to existing Via
+// header field to and from backend server.
+func TestH2H1NoVia(t *testing.T) {
+	st := newServerTester([]string{"--no-via"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "foo"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+		w.Header().Add("Via", "bar")
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2H1NoVia",
+		header: []hpack.HeaderField{
+			pair("via", "foo"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "bar"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestH2H1HostRewrite tests that server rewrites host header field
+func TestH2H1HostRewrite(t *testing.T) {
+	st := newServerTester(nil, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2H1HostRewrite",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := res.header.Get("request-host"), st.backendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}
+
+// TestH2H1NoHostRewrite tests that server does not rewrite host
+// header field
+func TestH2H1NoHostRewrite(t *testing.T) {
+	st := newServerTester([]string{"--no-host-rewrite"}, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2H1NoHostRewrite",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := res.header.Get("request-host"), st.frontendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}
+
 // TestH2H1BadRequestCL tests that server rejects request whose
 // content-length header field value does not match its request body
 // size.
@@ -241,9 +355,8 @@ func TestH2H1MultipleRequestCL(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Error st.http2() = %v", err)
 	}
-	want := 400
-	if got := res.status; got != want {
-		t.Errorf("status: %v; want %v", got, want)
+	if got, want := res.errCode, http2.ErrCodeProtocol; got != want {
+		t.Errorf("res.errCode: %v; want %v", got, want)
 	}
 }
 
@@ -264,9 +377,8 @@ func TestH2H1InvalidRequestCL(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Error st.http2() = %v", err)
 	}
-	want := 400
-	if got := res.status; got != want {
-		t.Errorf("status: %v; want %v", got, want)
+	if got, want := res.errCode, http2.ErrCodeProtocol; got != want {
+		t.Errorf("res.errCode: %v; want %v", got, want)
 	}
 }
 
@@ -317,16 +429,16 @@ func TestH2H1AssembleCookies(t *testing.T) {
 	}
 }
 
-// TestH2H1TETrailer tests that server accepts TE request header field
-// if it has trailer only.
-func TestH2H1TETrailer(t *testing.T) {
+// TestH2H1TETrailers tests that server accepts TE request header
+// field if it has trailers only.
+func TestH2H1TETrailers(t *testing.T) {
 	st := newServerTester(nil, t, noopHandler)
 	defer st.Close()
 
 	res, err := st.http2(requestParam{
-		name: "TestH2H1TETrailer",
+		name: "TestH2H1TETrailers",
 		header: []hpack.HeaderField{
-			pair("te", "trailer"),
+			pair("te", "trailers"),
 		},
 	})
 	if err != nil {
@@ -337,8 +449,8 @@ func TestH2H1TETrailer(t *testing.T) {
 	}
 }
 
-// TestH2H1TEGzip tests that server reset stream if TE request
-// header field contains gzip.
+// TestH2H1TEGzip tests that server resets stream if TE request header
+// field contains gzip.
 func TestH2H1TEGzip(t *testing.T) {
 	st := newServerTester(nil, t, func(w http.ResponseWriter, r *http.Request) {
 		t.Error("server should not forward bad request")
@@ -359,6 +471,9 @@ func TestH2H1TEGzip(t *testing.T) {
 	}
 }
 
+// TestH2H1SNI tests server's TLS SNI extension feature.  It must
+// choose appropriate certificate depending on the indicated
+// server_name from client.
 func TestH2H1SNI(t *testing.T) {
 	st := newServerTesterTLSConfig([]string{"--subcert=" + testDir + "/alt-server.key:" + testDir + "/alt-server.crt"}, t, noopHandler, &tls.Config{
 		ServerName: "alt-domain",
@@ -374,6 +489,37 @@ func TestH2H1SNI(t *testing.T) {
 	}
 }
 
+// TestH2H1ServerPush tests server push using Link header field from
+// backend server.
+func TestH2H1ServerPush(t *testing.T) {
+	st := newServerTester(nil, t, func(w http.ResponseWriter, r *http.Request) {
+		// only resources marked as rel=preload are pushed
+		w.Header().Add("Link", "</css/main.css>; rel=preload, </foo>, </css/theme.css>; rel=preload")
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2H1ServerPush",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+	if got, want := len(res.pushResponse), 2; got != want {
+		t.Fatalf("len(res.pushResponse): %v; want %v", got, want)
+	}
+	mainCSS := res.pushResponse[0]
+	if got, want := mainCSS.status, 200; got != want {
+		t.Errorf("mainCSS.status: %v; want %v", got, want)
+	}
+	themeCSS := res.pushResponse[1]
+	if got, want := themeCSS.status, 200; got != want {
+		t.Errorf("themeCSS.status: %v; want %v", got, want)
+	}
+}
+
 // TestH2H1GracefulShutdown tests graceful shutdown.
 func TestH2H1GracefulShutdown(t *testing.T) {
 	st := newServerTester(nil, t, noopHandler)
@@ -466,9 +612,8 @@ func TestH2H2MultipleResponseCL(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Error st.http2() = %v", err)
 	}
-	want := 502
-	if got := res.status; got != want {
-		t.Errorf("status: %v; want %v", got, want)
+	if got, want := res.errCode, http2.ErrCodeInternal; got != want {
+		t.Errorf("res.errCode: %v; want %v", got, want)
 	}
 }
 
@@ -487,9 +632,8 @@ func TestH2H2InvalidResponseCL(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Error st.http2() = %v", err)
 	}
-	want := 502
-	if got := res.status; got != want {
-		t.Errorf("status: %v; want %v", got, want)
+	if got, want := res.errCode, http2.ErrCodeInternal; got != want {
+		t.Errorf("res.errCode: %v; want %v", got, want)
 	}
 }
 
@@ -513,3 +657,46 @@ func TestH2H2ConnectFailure(t *testing.T) {
 		t.Errorf("status: %v; want %v", got, want)
 	}
 }
+
+// TestH2H2HostRewrite tests that server rewrites host header field
+func TestH2H2HostRewrite(t *testing.T) {
+	st := newServerTester([]string{"--http2-bridge"}, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2H2HostRewrite",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := res.header.Get("request-host"), st.backendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}
+
+// TestH2H2NoHostRewrite tests that server does not rewrite host
+// header field
+func TestH2H2NoHostRewrite(t *testing.T) {
+	st := newServerTester([]string{"--http2-bridge", "--no-host-rewrite"}, t, func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Add("request-host", r.Host)
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2H2NoHostRewrite",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+	if got, want := res.header.Get("request-host"), st.frontendHost; got != want {
+		t.Errorf("request-host: %v; want %v", got, want)
+	}
+}

integration-tests/nghttpx_spdy_test.go

@@ -99,6 +99,77 @@ func TestS3H1InvalidRequestCL(t *testing.T) {
 	}
 }
 
+// TestS3H1GenerateVia tests that server generates Via header field to and
+// from backend server.
+func TestS3H1GenerateVia(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "1.1 nghttpx"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1GenerateVia",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "1.1 nghttpx"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1AppendVia tests that server adds value to existing Via
+// header field to and from backend server.
+func TestS3H1AppendVia(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "foo, 1.1 nghttpx"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+		w.Header().Add("Via", "bar")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1AppendVia",
+		header: []hpack.HeaderField{
+			pair("via", "foo"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "bar, 1.1 nghttpx"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1NoVia tests that server does not add value to existing Via
+// header field to and from backend server.
+func TestS3H1NoVia(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--no-via"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "foo"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+		w.Header().Add("Via", "bar")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1NoVia",
+		header: []hpack.HeaderField{
+			pair("via", "foo"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "bar"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
 // TestS3H2ConnectFailure tests that server handles the situation that
 // connection attempt to HTTP/2 backend failed.
 func TestS3H2ConnectFailure(t *testing.T) {

integration-tests/server_tester.go

@@ -17,6 +17,7 @@ import (
 	"net/http/httptest"
 	"net/url"
 	"os/exec"
+	"sort"
 	"strconv"
 	"strings"
 	"testing"
@@ -42,6 +43,8 @@ type serverTester struct {
 	url           string    // test frontend server URL
 	t             *testing.T
 	ts            *httptest.Server // backend server
+	frontendHost  string           // frontend server host
+	backendHost   string           // backend server host
 	conn          net.Conn         // connection to frontend server
 	h2PrefaceSent bool             // HTTP/2 preface was sent in conn
 	nextStreamID  uint32           // next stream ID
@@ -124,6 +127,8 @@ func newServerTesterInternal(args []string, t *testing.T, handler http.HandlerFu
 		t:            t,
 		ts:           ts,
 		url:          fmt.Sprintf("%v://%v", scheme, authority),
+		frontendHost: fmt.Sprintf("127.0.0.1:%v", serverPort),
+		backendHost:  backendURL.Host,
 		nextStreamID: 1,
 		authority:    authority,
 		frCh:         make(chan http2.Frame),
@@ -407,7 +412,6 @@ loop:
 }
 
 func (st *serverTester) http2(rp requestParam) (*serverResponse, error) {
-	res := &serverResponse{}
 	st.headerBlkBuf.Reset()
 	st.header = make(http.Header)
 
@@ -430,6 +434,13 @@ func (st *serverTester) http2(rp requestParam) (*serverResponse, error) {
 		}
 	}
 
+	res := &serverResponse{
+		streamID: id,
+	}
+
+	streams := make(map[uint32]*serverResponse)
+	streams[id] = res
+
 	method := "GET"
 	if rp.method != "" {
 		method = rp.method
@@ -489,34 +500,53 @@ loop:
 			if err != nil {
 				return res, err
 			}
-			if f.FrameHeader.StreamID != id {
+			sr, ok := streams[f.FrameHeader.StreamID]
+			if !ok {
 				st.header = make(http.Header)
 				break
 			}
-			res.header = cloneHeader(st.header)
+			sr.header = cloneHeader(st.header)
 			var status int
-			status, err = strconv.Atoi(res.header.Get(":status"))
+			status, err = strconv.Atoi(sr.header.Get(":status"))
 			if err != nil {
 				return res, fmt.Errorf("Error parsing status code: %v", err)
 			}
-			res.status = status
+			sr.status = status
 			if f.StreamEnded() {
-				break loop
+				if streamEnded(res, streams, sr) {
+					break loop
+				}
 			}
+		case *http2.PushPromiseFrame:
+			_, err := st.dec.Write(f.HeaderBlockFragment())
+			if err != nil {
+				return res, err
+			}
+			sr := &serverResponse{
+				streamID:  f.PromiseID,
+				reqHeader: cloneHeader(st.header),
+			}
+			streams[sr.streamID] = sr
 		case *http2.DataFrame:
-			if f.FrameHeader.StreamID != id {
+			sr, ok := streams[f.FrameHeader.StreamID]
+			if !ok {
 				break
 			}
-			res.body = append(res.body, f.Data()...)
+			sr.body = append(sr.body, f.Data()...)
 			if f.StreamEnded() {
-				break loop
+				if streamEnded(res, streams, sr) {
+					break loop
+				}
 			}
 		case *http2.RSTStreamFrame:
-			if f.FrameHeader.StreamID != id {
+			sr, ok := streams[f.FrameHeader.StreamID]
+			if !ok {
 				break
 			}
-			res.errCode = f.ErrCode
-			break loop
+			sr.errCode = f.ErrCode
+			if streamEnded(res, streams, sr) {
+				break loop
+			}
 		case *http2.GoAwayFrame:
 			if f.ErrCode == http2.ErrCodeNo {
 				break
@@ -531,21 +561,46 @@ loop:
 			if err := st.fr.WriteSettingsAck(); err != nil {
 				return res, err
 			}
-			// TODO handle PUSH_PROMISE as well, since it alters HPACK context
 		}
 	}
+	sort.Sort(ByStreamID(res.pushResponse))
 	return res, nil
 }
 
+func streamEnded(mainSr *serverResponse, streams map[uint32]*serverResponse, sr *serverResponse) bool {
+	delete(streams, sr.streamID)
+	if mainSr.streamID != sr.streamID {
+		mainSr.pushResponse = append(mainSr.pushResponse, sr)
+	}
+	return len(streams) == 0
+}
+
 type serverResponse struct {
 	status            int                  // HTTP status code
 	header            http.Header          // response header fields
 	body              []byte               // response body
+	streamID          uint32               // stream ID in HTTP/2
 	errCode           http2.ErrCode        // error code received in HTTP/2 RST_STREAM or GOAWAY
 	connErr           bool                 // true if HTTP/2 connection error
 	spdyGoAwayErrCode spdy.GoAwayStatus    // status code received in SPDY RST_STREAM
 	spdyRstErrCode    spdy.RstStreamStatus // status code received in SPDY GOAWAY
 	connClose         bool                 // Conection: close is included in response header in HTTP/1 test
+	reqHeader         http.Header          // http request header, currently only sotres pushed request header
+	pushResponse      []*serverResponse    // pushed response
+}
+
+type ByStreamID []*serverResponse
+
+func (b ByStreamID) Len() int {
+	return len(b)
+}
+
+func (b ByStreamID) Swap(i, j int) {
+	b[i], b[j] = b[j], b[i]
+}
+
+func (b ByStreamID) Less(i, j int) bool {
+	return b[i].streamID < b[j].streamID
 }
 
 func cloneHeader(h http.Header) http.Header {

lib/Makefile.am

@@ -45,7 +45,8 @@ OBJECTS = nghttp2_pq.c nghttp2_map.c nghttp2_queue.c \
 	nghttp2_priority_spec.c \
 	nghttp2_option.c \
 	nghttp2_callbacks.c \
-	nghttp2_mem.c
+	nghttp2_mem.c \
+	nghttp2_http.c
 
 HFILES = nghttp2_pq.h nghttp2_int.h nghttp2_map.h nghttp2_queue.h \
 	nghttp2_frame.h \
@@ -58,7 +59,8 @@ HFILES = nghttp2_pq.h nghttp2_int.h nghttp2_map.h nghttp2_queue.h \
 	nghttp2_priority_spec.h \
 	nghttp2_option.h \
 	nghttp2_callbacks.h \
-	nghttp2_mem.h
+	nghttp2_mem.h \
+	nghttp2_http.h
 
 libnghttp2_la_SOURCES = $(HFILES) $(OBJECTS)
 libnghttp2_la_LDFLAGS = -no-undefined \

lib/includes/nghttp2/nghttp2.h

@@ -49,6 +49,24 @@ extern "C" {
  */
 #define NGHTTP2_PROTO_VERSION_ID_LEN 5
 
+/**
+ * @macro
+ *
+ * The seriazlied form of ALPN protocol identifier this library
+ * supports.  Notice that first byte is the length of following
+ * protocol identifier.  This is the same wire format of `TLS ALPN
+ * extension <https://tools.ietf.org/html/rfc7301>`_.  This is useful
+ * to process incoming ALPN tokens in wire format.
+ */
+#define NGHTTP2_PROTO_ALPN "\x5h2-14"
+
+/**
+ * @macro
+ *
+ * The length of :macro:`NGHTTP2_PROTO_ALPN`.
+ */
+#define NGHTTP2_PROTO_ALPN_LEN (sizeof(NGHTTP2_PROTO_ALPN) - 1)
+
 /**
  * @macro
  *
@@ -330,6 +348,11 @@ typedef enum {
    * `nghttp2_session_terminate_session()` is called.
    */
   NGHTTP2_ERR_SESSION_CLOSING = -530,
+  /**
+   * Invalid HTTP header field was received and stream is going to be
+   * closed.
+   */
+  NGHTTP2_ERR_HTTP_HEADER = -531,
   /**
    * The errors < :enum:`NGHTTP2_ERR_FATAL` mean that the library is
    * under unexpected condition and processing was terminated (e.g.,
@@ -365,7 +388,9 @@ typedef enum {
    */
   NGHTTP2_NV_FLAG_NONE = 0,
   /**
-   * Indicates that this name/value pair must not be indexed.
+   * Indicates that this name/value pair must not be indexed ("Literal
+   * Header Field never Indexed" representation must be used in HPACK
+   * encoding).  Other implementation calls this bit as "sensitive".
    */
   NGHTTP2_NV_FLAG_NO_INDEX = 0x01
 } nghttp2_nv_flag;
@@ -1304,6 +1329,9 @@ typedef int (*nghttp2_on_frame_send_callback)(nghttp2_session *session,
  * `nghttp2_session_recv()` and `nghttp2_session_send()` functions
  * immediately return :enum:`NGHTTP2_ERR_CALLBACK_FAILURE`.
  *
+ * `nghttp2_session_get_stream_user_data()` can be used to get
+ * associated data.
+ *
  * To set this callback to :type:`nghttp2_session_callbacks`, use
  * `nghttp2_session_callbacks_set_on_frame_not_send_callback()`.
  */
@@ -1352,6 +1380,27 @@ typedef int (*nghttp2_on_stream_close_callback)(nghttp2_session *session,
  * not need to care about that because the header name/value pairs are
  * emitted transparently regardless of CONTINUATION frames.
  *
+ * The server applications probably create an object to store
+ * information about new stream if ``frame->hd.type ==
+ * NGHTTP2_HEADERS`` and ``frame->headers.cat ==
+ * NGHTTP2_HCAT_REQUEST``.  If |session| is configured as server side,
+ * ``frame->headers.cat`` is either ``NGHTTP2_HCAT_REQUEST``
+ * containing request headers or ``NGHTTP2_HCAT_HEADERS`` containing
+ * trailer headers and never get PUSH_PROMISE in this callback.
+ *
+ * For the client applications, ``frame->hd.type`` is either
+ * ``NGHTTP2_HEADERS`` or ``NGHTTP2_PUSH_PROMISE``.  In case of
+ * ``NGHTTP2_HEADERS``, ``frame->headers.cat ==
+ * NGHTTP2_HCAT_RESPONSE`` means that it is the first response
+ * headers, but it may be non-final response which is indicated by 1xx
+ * status code.  In this case, there may be zero or more HEADERS frame
+ * with ``frame->headers.cat == NGHTTP2_HCAT_HEADERS`` which has
+ * non-final response code and finally client gets exactly one HEADERS
+ * frame with ``frame->headers.cat == NGHTTP2_HCAT_HEADERS``
+ * containing final response headers (non-1xx status code).  The
+ * trailer headers also has ``frame->headers.cat ==
+ * NGHTTP2_HCAT_HEADERS`` which does not containg any status code.
+ *
  * The implementation of this function must return 0 if it succeeds or
  * :enum:`NGHTTP2_ERR_CALLBACK_FAILURE`.  If nonzero value other than
  * :enum:`NGHTTP2_ERR_CALLBACK_FAILURE` is returned, it is treated as
@@ -1377,7 +1426,8 @@ typedef int (*nghttp2_on_begin_headers_callback)(nghttp2_session *session,
  *
  * If :enum:`NGHTTP2_NV_FLAG_NO_INDEX` is set in |flags|, the receiver
  * must not index this name/value pair when forwarding it to the next
- * hop.
+ * hop.  More specifically, "Literal Header Field never Indexed"
+ * representation must be used in HPACK encoding.
  *
  * When this callback is invoked, ``frame->hd.type`` is either
  * :enum:`NGHTTP2_HEADERS` or :enum:`NGHTTP2_PUSH_PROMISE`.  After all
@@ -1387,21 +1437,14 @@ typedef int (*nghttp2_on_begin_headers_callback)(nghttp2_session *session,
  * :type:`nghttp2_on_frame_recv_callback` for the |frame| will not be
  * invoked.
  *
- * The |name| may be ``NULL`` if the |namelen| is 0.  The same thing
- * can be said about the |value|.
+ * The |value| may be ``NULL`` if the |valuelen| is 0.
  *
- * Please note that nghttp2 library does not perform any validity
- * check against the |name| and the |value|.  For example, the
- * |namelen| could be 0, and/or the |value| contains ``0x0a`` or
- * ``0x0d``.  The application must check them if it matters.  The
- * helper function `nghttp2_check_header_name()` and
- * `nghttp2_check_header_value()` provide simple validation against
- * HTTP2 header field construction rule.
- *
- * HTTP/2 specification requires that pseudo header fields (header
- * field starting with ':') must appear in front of regular header
- * fields.  The library does not validate this requirement.  The
- * application must check them if it matters.
+ * Please note that unless `nghttp2_option_set_no_http_messaging()` is
+ * used, nghttp2 library does perform validation against the |name|
+ * and the |value| using `nghttp2_check_header_name()` and
+ * `nghttp2_check_header_value()`.  In addition to this, nghttp2
+ * performs vaidation based on HTTP Messaging rule, which is briefly
+ * explained in `HTTP Messaging`_ section.
  *
  * If the application uses `nghttp2_session_mem_recv()`, it can return
  * :enum:`NGHTTP2_ERR_PAUSE` to make `nghttp2_session_mem_recv()`
@@ -1869,6 +1912,18 @@ void nghttp2_option_set_peer_max_concurrent_streams(nghttp2_option *option,
  */
 void nghttp2_option_set_recv_client_preface(nghttp2_option *option, int val);
 
+/**
+ * @function
+ *
+ * By default, nghttp2 library enforces subset of HTTP Messaging rules
+ * described in `HTTP/2 specification, section 8
+ * <https://tools.ietf.org/html/draft-ietf-httpbis-http2-17#section-8>`_.
+ * See `HTTP Messaging`_ section for details.  For those applications
+ * who use nghttp2 library as non-HTTP use, give nonzero to |val| to
+ * disable this enforcement.
+ */
+void nghttp2_option_set_no_http_messaging(nghttp2_option *option, int val);
+
 /**
  * @function
  *
@@ -3016,6 +3071,9 @@ int nghttp2_submit_settings(nghttp2_session *session, uint8_t flags,
  *
  * The client side is not allowed to use this function.
  *
+ * To submit response headers and data, use
+ * `nghttp2_submit_response()`.
+ *
  * This function returns assigned promised stream ID if it succeeds,
  * or one of the following negative error codes:
  *

lib/nghttp2_frame.h

@@ -55,10 +55,8 @@
 /* Number of inbound buffer */
 #define NGHTTP2_FRAMEBUF_MAX_NUM 5
 
-/* The default length of DATA frame payload. This should be small enough
- * for the data payload and the header to fit into 1 TLS record */
-#define NGHTTP2_DATA_PAYLOADLEN                                                \
-  ((NGHTTP2_MAX_FRAME_SIZE_MIN) - (NGHTTP2_FRAME_HDLEN))
+/* The default length of DATA frame payload. */
+#define NGHTTP2_DATA_PAYLOADLEN NGHTTP2_MAX_FRAME_SIZE_MIN
 
 /* Maximum headers payload length, calculated in compressed form.
    This applies to transmission only. */

lib/nghttp2_hd.c

@@ -37,8 +37,8 @@
 #define MAKE_STATIC_ENT(I, N, V, NH, VH)                                       \
   {                                                                            \
     {                                                                          \
-      { (uint8_t *) N, (uint8_t *)V, sizeof(N) - 1, sizeof(V) - 1, 0 }         \
-      , NH, VH, 1, NGHTTP2_HD_FLAG_NONE                                        \
+      { (uint8_t *)(N), (uint8_t *)(V), sizeof((N)) - 1, sizeof((V)) - 1, 0 }  \
+      , (NH), (VH), 1, NGHTTP2_HD_FLAG_NONE                                    \
     }                                                                          \
     , I                                                                        \
   }

lib/nghttp2_helper.c

@@ -267,8 +267,6 @@ const char *nghttp2_strerror(int error_code) {
     return "Invalid stream state";
   case NGHTTP2_ERR_DEFERRED_DATA_EXIST:
     return "Another DATA frame has already been deferred";
-  case NGHTTP2_ERR_SESSION_CLOSING:
-    return "The current session is closing";
   case NGHTTP2_ERR_START_STREAM_NOT_ALLOWED:
     return "request HEADERS is not allowed";
   case NGHTTP2_ERR_GOAWAY_ALREADY_SENT:
@@ -295,6 +293,10 @@ const char *nghttp2_strerror(int error_code) {
     return "Server push is disabled by peer";
   case NGHTTP2_ERR_DATA_EXIST:
     return "DATA frame already exists";
+  case NGHTTP2_ERR_SESSION_CLOSING:
+    return "The current session is closing";
+  case NGHTTP2_ERR_HTTP_HEADER:
+    return "Invalid HTTP header field was received";
   case NGHTTP2_ERR_NOMEM:
     return "Out of memory";
   case NGHTTP2_ERR_CALLBACK_FAILURE:

lib/nghttp2_http.c

@@ -0,0 +1,548 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2015 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "nghttp2_http.h"
+
+#include <string.h>
+#include <assert.h>
+#include <stdio.h>
+
+static int memeq(const void *a, const void *b, size_t n) {
+  return memcmp(a, b, n) == 0;
+}
+
+#define streq(A, B, N) ((sizeof((A)) - 1) == (N) && memeq((A), (B), (N)))
+
+static char downcase(char c) {
+  return 'A' <= c && c <= 'Z' ? (c - 'A' + 'a') : c;
+}
+
+static int memieq(const void *a, const void *b, size_t n) {
+  size_t i;
+  const uint8_t *aa = a, *bb = b;
+
+  for (i = 0; i < n; ++i) {
+    if (downcase(aa[i]) != downcase(bb[i])) {
+      return 0;
+    }
+  }
+  return 1;
+}
+
+#define strieq(A, B, N) ((sizeof((A)) - 1) == (N) && memieq((A), (B), (N)))
+
+typedef enum {
+  NGHTTP2_TOKEN__AUTHORITY,
+  NGHTTP2_TOKEN__METHOD,
+  NGHTTP2_TOKEN__PATH,
+  NGHTTP2_TOKEN__SCHEME,
+  NGHTTP2_TOKEN__STATUS,
+  NGHTTP2_TOKEN_CONNECTION,
+  NGHTTP2_TOKEN_CONTENT_LENGTH,
+  NGHTTP2_TOKEN_HOST,
+  NGHTTP2_TOKEN_KEEP_ALIVE,
+  NGHTTP2_TOKEN_PROXY_CONNECTION,
+  NGHTTP2_TOKEN_TE,
+  NGHTTP2_TOKEN_TRANSFER_ENCODING,
+  NGHTTP2_TOKEN_UPGRADE,
+  NGHTTP2_TOKEN_MAXIDX,
+} nghttp2_token;
+
+/*
+ * This function was generated by genlibtokenlookup.py.  Inspired by
+ * h2o header lookup.  https://github.com/h2o/h2o
+ */
+static int lookup_token(const uint8_t *name, size_t namelen) {
+  switch (namelen) {
+  case 2:
+    switch (name[namelen - 1]) {
+    case 'e':
+      if (streq("t", name, 1)) {
+        return NGHTTP2_TOKEN_TE;
+      }
+      break;
+    }
+    break;
+  case 4:
+    switch (name[namelen - 1]) {
+    case 't':
+      if (streq("hos", name, 3)) {
+        return NGHTTP2_TOKEN_HOST;
+      }
+      break;
+    }
+    break;
+  case 5:
+    switch (name[namelen - 1]) {
+    case 'h':
+      if (streq(":pat", name, 4)) {
+        return NGHTTP2_TOKEN__PATH;
+      }
+      break;
+    }
+    break;
+  case 7:
+    switch (name[namelen - 1]) {
+    case 'd':
+      if (streq(":metho", name, 6)) {
+        return NGHTTP2_TOKEN__METHOD;
+      }
+      break;
+    case 'e':
+      if (streq(":schem", name, 6)) {
+        return NGHTTP2_TOKEN__SCHEME;
+      }
+      if (streq("upgrad", name, 6)) {
+        return NGHTTP2_TOKEN_UPGRADE;
+      }
+      break;
+    case 's':
+      if (streq(":statu", name, 6)) {
+        return NGHTTP2_TOKEN__STATUS;
+      }
+      break;
+    }
+    break;
+  case 10:
+    switch (name[namelen - 1]) {
+    case 'e':
+      if (streq("keep-aliv", name, 9)) {
+        return NGHTTP2_TOKEN_KEEP_ALIVE;
+      }
+      break;
+    case 'n':
+      if (streq("connectio", name, 9)) {
+        return NGHTTP2_TOKEN_CONNECTION;
+      }
+      break;
+    case 'y':
+      if (streq(":authorit", name, 9)) {
+        return NGHTTP2_TOKEN__AUTHORITY;
+      }
+      break;
+    }
+    break;
+  case 14:
+    switch (name[namelen - 1]) {
+    case 'h':
+      if (streq("content-lengt", name, 13)) {
+        return NGHTTP2_TOKEN_CONTENT_LENGTH;
+      }
+      break;
+    }
+    break;
+  case 16:
+    switch (name[namelen - 1]) {
+    case 'n':
+      if (streq("proxy-connectio", name, 15)) {
+        return NGHTTP2_TOKEN_PROXY_CONNECTION;
+      }
+      break;
+    }
+    break;
+  case 17:
+    switch (name[namelen - 1]) {
+    case 'g':
+      if (streq("transfer-encodin", name, 16)) {
+        return NGHTTP2_TOKEN_TRANSFER_ENCODING;
+      }
+      break;
+    }
+    break;
+  }
+  return -1;
+}
+
+static int64_t parse_uint(const uint8_t *s, size_t len) {
+  int64_t n = 0;
+  size_t i;
+  if (len == 0) {
+    return -1;
+  }
+  for (i = 0; i < len; ++i) {
+    if ('0' <= s[i] && s[i] <= '9') {
+      if (n > INT64_MAX / 10) {
+        return -1;
+      }
+      n *= 10;
+      if (n > INT64_MAX - (s[i] - '0')) {
+        return -1;
+      }
+      n += s[i] - '0';
+      continue;
+    }
+    return -1;
+  }
+  return n;
+}
+
+static int lws(const uint8_t *s, size_t n) {
+  size_t i;
+  for (i = 0; i < n; ++i) {
+    if (s[i] != ' ' && s[i] != '\t') {
+      return 0;
+    }
+  }
+  return 1;
+}
+
+static int check_pseudo_header(nghttp2_stream *stream, const nghttp2_nv *nv,
+                               int flag) {
+  if (stream->http_flags & flag) {
+    return 0;
+  }
+  if (lws(nv->value, nv->valuelen)) {
+    return 0;
+  }
+  stream->http_flags |= flag;
+  return 1;
+}
+
+static int expect_response_body(nghttp2_stream *stream) {
+  return (stream->http_flags & NGHTTP2_HTTP_FLAG_METH_HEAD) == 0 &&
+         stream->status_code / 100 != 1 && stream->status_code != 304 &&
+         stream->status_code != 204;
+}
+
+static int http_request_on_header(nghttp2_stream *stream, nghttp2_nv *nv,
+                                  int trailer) {
+  int token;
+
+  if (nv->name[0] == ':') {
+    if (trailer ||
+        (stream->http_flags & NGHTTP2_HTTP_FLAG_PSEUDO_HEADER_DISALLOWED)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+  }
+
+  token = lookup_token(nv->name, nv->namelen);
+
+  switch (token) {
+  case NGHTTP2_TOKEN__AUTHORITY:
+    if (!check_pseudo_header(stream, nv, NGHTTP2_HTTP_FLAG__AUTHORITY)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  case NGHTTP2_TOKEN__METHOD:
+    if (!check_pseudo_header(stream, nv, NGHTTP2_HTTP_FLAG__METHOD)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    if (streq("HEAD", nv->value, nv->valuelen)) {
+      stream->http_flags |= NGHTTP2_HTTP_FLAG_METH_HEAD;
+    } else if (streq("CONNECT", nv->value, nv->valuelen)) {
+      if (stream->stream_id % 2 == 0) {
+        /* we won't allow CONNECT for push */
+        return NGHTTP2_ERR_HTTP_HEADER;
+      }
+      stream->http_flags |= NGHTTP2_HTTP_FLAG_METH_CONNECT;
+      if (stream->http_flags &
+          (NGHTTP2_HTTP_FLAG__PATH | NGHTTP2_HTTP_FLAG__SCHEME)) {
+        return NGHTTP2_ERR_HTTP_HEADER;
+      }
+    }
+    break;
+  case NGHTTP2_TOKEN__PATH:
+    if (stream->http_flags & NGHTTP2_HTTP_FLAG_METH_CONNECT) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    if (!check_pseudo_header(stream, nv, NGHTTP2_HTTP_FLAG__PATH)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  case NGHTTP2_TOKEN__SCHEME:
+    if (stream->http_flags & NGHTTP2_HTTP_FLAG_METH_CONNECT) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    if (!check_pseudo_header(stream, nv, NGHTTP2_HTTP_FLAG__SCHEME)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  case NGHTTP2_TOKEN_HOST:
+    if (!check_pseudo_header(stream, nv, NGHTTP2_HTTP_FLAG_HOST)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  case NGHTTP2_TOKEN_CONTENT_LENGTH: {
+    if (stream->content_length != -1) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    stream->content_length = parse_uint(nv->value, nv->valuelen);
+    if (stream->content_length == -1) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  }
+  /* disallowed header fields */
+  case NGHTTP2_TOKEN_CONNECTION:
+  case NGHTTP2_TOKEN_KEEP_ALIVE:
+  case NGHTTP2_TOKEN_PROXY_CONNECTION:
+  case NGHTTP2_TOKEN_TRANSFER_ENCODING:
+  case NGHTTP2_TOKEN_UPGRADE:
+    return NGHTTP2_ERR_HTTP_HEADER;
+  case NGHTTP2_TOKEN_TE:
+    if (!strieq("trailers", nv->value, nv->valuelen)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  default:
+    if (nv->name[0] == ':') {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+  }
+
+  if (nv->name[0] != ':') {
+    stream->http_flags |= NGHTTP2_HTTP_FLAG_PSEUDO_HEADER_DISALLOWED;
+  }
+
+  return 0;
+}
+
+static int http_response_on_header(nghttp2_stream *stream, nghttp2_nv *nv,
+                                   int trailer) {
+  int token;
+
+  if (nv->name[0] == ':') {
+    if (trailer ||
+        (stream->http_flags & NGHTTP2_HTTP_FLAG_PSEUDO_HEADER_DISALLOWED)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+  }
+
+  token = lookup_token(nv->name, nv->namelen);
+
+  switch (token) {
+  case NGHTTP2_TOKEN__STATUS: {
+    if (!check_pseudo_header(stream, nv, NGHTTP2_HTTP_FLAG__STATUS)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    if (nv->valuelen != 3) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    stream->status_code = parse_uint(nv->value, nv->valuelen);
+    if (stream->status_code == -1) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  }
+  case NGHTTP2_TOKEN_CONTENT_LENGTH: {
+    if (stream->content_length != -1) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    stream->content_length = parse_uint(nv->value, nv->valuelen);
+    if (stream->content_length == -1) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  }
+  /* disallowed header fields */
+  case NGHTTP2_TOKEN_CONNECTION:
+  case NGHTTP2_TOKEN_KEEP_ALIVE:
+  case NGHTTP2_TOKEN_PROXY_CONNECTION:
+  case NGHTTP2_TOKEN_TRANSFER_ENCODING:
+  case NGHTTP2_TOKEN_UPGRADE:
+    return NGHTTP2_ERR_HTTP_HEADER;
+  case NGHTTP2_TOKEN_TE:
+    if (!strieq("trailers", nv->value, nv->valuelen)) {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    break;
+  default:
+    if (nv->name[0] == ':') {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+  }
+
+  if (nv->name[0] != ':') {
+    stream->http_flags |= NGHTTP2_HTTP_FLAG_PSEUDO_HEADER_DISALLOWED;
+  }
+
+  return 0;
+}
+
+int nghttp2_http_on_header(nghttp2_session *session, nghttp2_stream *stream,
+                           nghttp2_frame *frame, nghttp2_nv *nv, int trailer) {
+  /* We are strict for pseudo header field.  One bad character should
+     lead to fail.  OTOH, we should be a bit forgiving for regular
+     headers, since existing public internet has so much illegal
+     headers floating around and if we kill the stream because of
+     this, we may disrupt many web sites and/or libraries.  So we
+     become conservative here, and just ignore those illegal regular
+     headers. */
+  if (!nghttp2_check_header_name(nv->name, nv->namelen)) {
+    size_t i;
+    if (nv->namelen > 0 && nv->name[0] == ':') {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    /* header field name must be lower-cased without exception */
+    for (i = 0; i < nv->namelen; ++i) {
+      char c = nv->name[i];
+      if ('A' <= c && c <= 'Z') {
+        return NGHTTP2_ERR_HTTP_HEADER;
+      }
+    }
+    /* When ignoring regular headers, we set this flag so that we
+       still enforce header field ordering rule for pseudo header
+       fields. */
+    stream->http_flags |= NGHTTP2_HTTP_FLAG_PSEUDO_HEADER_DISALLOWED;
+    return NGHTTP2_ERR_IGN_HTTP_HEADER;
+  }
+
+  if (!nghttp2_check_header_value(nv->value, nv->valuelen)) {
+    assert(nv->namelen > 0);
+    if (nv->name[0] == ':') {
+      return NGHTTP2_ERR_HTTP_HEADER;
+    }
+    /* When ignoring regular headers, we set this flag so that we
+       still enforce header field ordering rule for pseudo header
+       fields. */
+    stream->http_flags |= NGHTTP2_HTTP_FLAG_PSEUDO_HEADER_DISALLOWED;
+    return NGHTTP2_ERR_IGN_HTTP_HEADER;
+  }
+
+  if (session->server || frame->hd.type == NGHTTP2_PUSH_PROMISE) {
+    return http_request_on_header(stream, nv, trailer);
+  }
+
+  return http_response_on_header(stream, nv, trailer);
+}
+
+int nghttp2_http_on_request_headers(nghttp2_stream *stream,
+                                    nghttp2_frame *frame) {
+  if (stream->http_flags & NGHTTP2_HTTP_FLAG_METH_CONNECT) {
+    if ((stream->http_flags & NGHTTP2_HTTP_FLAG__AUTHORITY) == 0) {
+      return -1;
+    }
+    stream->content_length = -1;
+  } else if ((stream->http_flags & NGHTTP2_HTTP_FLAG_REQ_HEADERS) !=
+                 NGHTTP2_HTTP_FLAG_REQ_HEADERS ||
+             (stream->http_flags &
+              (NGHTTP2_HTTP_FLAG__AUTHORITY | NGHTTP2_HTTP_FLAG_HOST)) == 0) {
+    return -1;
+  }
+
+  if (frame->hd.type == NGHTTP2_PUSH_PROMISE) {
+    /* we are going to reuse data fields for upcoming response.  Clear
+       them now, except for method flags. */
+    stream->http_flags &= NGHTTP2_HTTP_FLAG_METH_ALL;
+    stream->content_length = -1;
+  }
+
+  return 0;
+}
+
+int nghttp2_http_on_response_headers(nghttp2_stream *stream) {
+  if ((stream->http_flags & NGHTTP2_HTTP_FLAG__STATUS) == 0) {
+    return -1;
+  }
+
+  if (stream->status_code / 100 == 1) {
+    /* non-final response */
+    stream->http_flags = (stream->http_flags & NGHTTP2_HTTP_FLAG_METH_ALL) |
+                         NGHTTP2_HTTP_FLAG_EXPECT_FINAL_RESPONSE;
+    stream->content_length = -1;
+    stream->status_code = -1;
+    return 0;
+  }
+
+  stream->http_flags &= ~NGHTTP2_HTTP_FLAG_EXPECT_FINAL_RESPONSE;
+
+  if (!expect_response_body(stream)) {
+    stream->content_length = 0;
+  } else if (stream->http_flags & NGHTTP2_HTTP_FLAG_METH_CONNECT) {
+    stream->content_length = -1;
+  }
+
+  return 0;
+}
+
+int nghttp2_http_on_trailer_headers(nghttp2_stream *stream _U_,
+                                    nghttp2_frame *frame) {
+  if ((frame->hd.flags & NGHTTP2_FLAG_END_STREAM) == 0) {
+    return -1;
+  }
+
+  return 0;
+}
+
+int nghttp2_http_on_remote_end_stream(nghttp2_stream *stream) {
+  if (stream->http_flags & NGHTTP2_HTTP_FLAG_EXPECT_FINAL_RESPONSE) {
+    return -1;
+  }
+
+  if (stream->content_length != -1 &&
+      stream->content_length != stream->recv_content_length) {
+    return -1;
+  }
+
+  return 0;
+}
+
+int nghttp2_http_on_data_chunk(nghttp2_stream *stream, size_t n) {
+  stream->recv_content_length += n;
+
+  if ((stream->http_flags & NGHTTP2_HTTP_FLAG_EXPECT_FINAL_RESPONSE) ||
+      (stream->content_length != -1 &&
+       stream->recv_content_length > stream->content_length)) {
+    return -1;
+  }
+
+  return 0;
+}
+
+void nghttp2_http_record_request_method(nghttp2_stream *stream,
+                                        nghttp2_frame *frame) {
+  const nghttp2_nv *nva;
+  size_t nvlen;
+  size_t i;
+
+  switch (frame->hd.type) {
+  case NGHTTP2_HEADERS:
+    nva = frame->headers.nva;
+    nvlen = frame->headers.nvlen;
+    break;
+  case NGHTTP2_PUSH_PROMISE:
+    nva = frame->push_promise.nva;
+    nvlen = frame->push_promise.nvlen;
+    break;
+  default:
+    return;
+  }
+
+  /* TODO we should do this strictly. */
+  for (i = 0; i < nvlen; ++i) {
+    const nghttp2_nv *nv = &nva[i];
+    if (lookup_token(nv->name, nv->namelen) != NGHTTP2_TOKEN__METHOD) {
+      continue;
+    }
+    if (streq("CONNECT", nv->value, nv->valuelen)) {
+      stream->http_flags |= NGHTTP2_HTTP_FLAG_METH_CONNECT;
+      return;
+    }
+    if (streq("HEAD", nv->value, nv->valuelen)) {
+      stream->http_flags |= NGHTTP2_HTTP_FLAG_METH_HEAD;
+      return;
+    }
+    return;
+  }
+}

lib/nghttp2_http.h

@@ -0,0 +1,96 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2015 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef NGHTTP2_HTTP_H
+#define NGHTTP2_HTTP_H
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif /* HAVE_CONFIG_H */
+
+#include <nghttp2/nghttp2.h>
+#include "nghttp2_session.h"
+#include "nghttp2_stream.h"
+
+/*
+ * This function is called when HTTP header field |nv| in |frame| is
+ * received for |stream|.  This function will validate |nv| against
+ * the current state of stream.
+ *
+ * This function returns 0 if it succeeds, or one of the following
+ * negative error codes:
+ *
+ * NGHTTP2_ERR_HTTP_HEADER
+ *     Invalid HTTP header field was received.
+ * NGHTTP2_ERR_IGN_HTTP_HEADER
+ *     Invalid HTTP header field was received but it can be treated as
+ *     if it was not received because of compatibility reasons.
+ */
+int nghttp2_http_on_header(nghttp2_session *session, nghttp2_stream *stream,
+                           nghttp2_frame *frame, nghttp2_nv *nv, int trailer);
+
+/*
+ * This function is called when request header is received.  This
+ * function performs validation and returns 0 if it succeeds, or -1.
+ */
+int nghttp2_http_on_request_headers(nghttp2_stream *stream,
+                                    nghttp2_frame *frame);
+
+/*
+ * This function is called when response header is received.  This
+ * function performs validation and returns 0 if it succeeds, or -1.
+ */
+int nghttp2_http_on_response_headers(nghttp2_stream *stream);
+
+/*
+ * This function is called trailer header (for both request and
+ * response) is received.  This function performs validation and
+ * returns 0 if it succeeds, or -1.
+ */
+int nghttp2_http_on_trailer_headers(nghttp2_stream *stream,
+                                    nghttp2_frame *frame);
+
+/*
+ * This function is called when END_STREAM flag is seen in incoming
+ * frame.  This function performs validation and returns 0 if it
+ * succeeds, or -1.
+ */
+int nghttp2_http_on_remote_end_stream(nghttp2_stream *stream);
+
+/*
+ * This function is called when chunk of data is received.  This
+ * function performs validation and returns 0 if it succeeds, or -1.
+ */
+int nghttp2_http_on_data_chunk(nghttp2_stream *stream, size_t n);
+
+/*
+ * This function inspects header field in |frame| and records its
+ * method in stream->http_flags.  If frame->hd.type is neither
+ * NGHTTP2_HEADERS nor NGHTTP2_PUSH_PROMISE, this function does
+ * nothing.
+ */
+void nghttp2_http_record_request_method(nghttp2_stream *stream,
+                                        nghttp2_frame *frame);
+
+#endif /* NGHTTP2_HTTP_H */

lib/nghttp2_int.h

@@ -46,7 +46,12 @@ typedef int (*nghttp2_compar)(const void *lhs, const void *rhs);
 typedef enum {
   NGHTTP2_ERR_CREDENTIAL_PENDING = -101,
   NGHTTP2_ERR_IGN_HEADER_BLOCK = -103,
-  NGHTTP2_ERR_IGN_PAYLOAD = -104
+  NGHTTP2_ERR_IGN_PAYLOAD = -104,
+  /*
+   * Invalid HTTP header field was received but it can be treated as
+   * if it was not received because of compatibility reasons.
+   */
+  NGHTTP2_ERR_IGN_HTTP_HEADER = -105,
 } nghttp2_internal_error;
 
 #endif /* NGHTTP2_INT_H */

lib/nghttp2_npn.c

@@ -26,29 +26,32 @@
 
 #include <string.h>
 
+static int select_next_protocol(unsigned char **out, unsigned char *outlen,
+                                const unsigned char *in, unsigned int inlen,
+                                const char *key, unsigned int keylen) {
+  unsigned int i;
+  for (i = 0; i + keylen <= inlen; i += in [i] + 1) {
+    if (memcmp(&in[i], key, keylen) == 0) {
+      *out = (unsigned char *)&in[i + 1];
+      *outlen = in[i];
+      return 0;
+    }
+  }
+  return -1;
+}
+
+#define NGHTTP2_HTTP_1_1_ALPN "\x8http/1.1"
+#define NGHTTP2_HTTP_1_1_ALPN_LEN (sizeof(NGHTTP2_HTTP_1_1_ALPN) - 1)
+
 int nghttp2_select_next_protocol(unsigned char **out, unsigned char *outlen,
                                  const unsigned char *in, unsigned int inlen) {
-  int http_selected = 0;
-  unsigned int i = 0;
-  for (; i < inlen; i += in [i] + 1) {
-    if (in[i] == NGHTTP2_PROTO_VERSION_ID_LEN && i + 1 + in[i] <= inlen &&
-        memcmp(&in[i + 1], NGHTTP2_PROTO_VERSION_ID, in[i]) == 0) {
-      *out = (unsigned char *)&in[i + 1];
-      *outlen = in[i];
-      return 1;
-    }
-    if (in[i] == 8 && i + 1 + in[i] <= inlen &&
-        memcmp(&in[i + 1], "http/1.1", in[i]) == 0) {
-      http_selected = 1;
-      *out = (unsigned char *)&in[i + 1];
-      *outlen = in[i];
-      /* Go through to the next iteration, because "HTTP/2" may be
-         there */
-    }
+  if (select_next_protocol(out, outlen, in, inlen, NGHTTP2_PROTO_ALPN,
+                           NGHTTP2_PROTO_ALPN_LEN) == 0) {
+    return 1;
   }
-  if (http_selected) {
+  if (select_next_protocol(out, outlen, in, inlen, NGHTTP2_HTTP_1_1_ALPN,
+                           NGHTTP2_HTTP_1_1_ALPN_LEN) == 0) {
     return 0;
-  } else {
-    return -1;
   }
+  return -1;
 }

lib/nghttp2_option.c

@@ -51,3 +51,8 @@ void nghttp2_option_set_recv_client_preface(nghttp2_option *option, int val) {
   option->opt_set_mask |= NGHTTP2_OPT_RECV_CLIENT_PREFACE;
   option->recv_client_preface = val;
 }
+
+void nghttp2_option_set_no_http_messaging(nghttp2_option *option, int val) {
+  option->opt_set_mask |= NGHTTP2_OPT_NO_HTTP_MESSAGING;
+  option->no_http_messaging = val;
+}

lib/nghttp2_option.h

@@ -58,6 +58,7 @@ typedef enum {
    */
   NGHTTP2_OPT_PEER_MAX_CONCURRENT_STREAMS = 1 << 1,
   NGHTTP2_OPT_RECV_CLIENT_PREFACE = 1 << 2,
+  NGHTTP2_OPT_NO_HTTP_MESSAGING = 1 << 3,
 } nghttp2_option_flag;
 
 /**
@@ -81,6 +82,10 @@ struct nghttp2_option {
    * NGHTTP2_OPT_RECV_CLIENT_PREFACE
    */
   uint8_t recv_client_preface;
+  /**
+   * NGHTTP2_OPT_NO_HTTP_MESSAGING
+   */
+  uint8_t no_http_messaging;
 };
 
 #endif /* NGHTTP2_OPTION_H */

lib/nghttp2_outbound_item.h

@@ -44,6 +44,12 @@
 typedef struct {
   nghttp2_data_provider data_prd;
   void *stream_user_data;
+  /* error code when request HEADERS is canceled by RST_STREAM while
+     it is in queue. */
+  uint32_t error_code;
+  /* nonzero if request HEADERS is canceled.  The error code is stored
+     in |error_code|. */
+  uint8_t canceled;
   /* nonzero if this item should be attached to stream object to make
      it under priority control */
   uint8_t attach_stream;

lib/nghttp2_pq.c

@@ -130,3 +130,17 @@ void nghttp2_pq_update(nghttp2_pq *pq, nghttp2_pq_item_cb fun, void *arg) {
     }
   }
 }
+
+int nghttp2_pq_each(nghttp2_pq *pq, nghttp2_pq_item_cb fun, void *arg) {
+  size_t i;
+
+  if (pq->length == 0) {
+    return 0;
+  }
+  for (i = 0; i < pq->length; ++i) {
+    if ((*fun)(pq->q[i], arg)) {
+      return 1;
+    }
+  }
+  return 0;
+}

lib/nghttp2_pq.h

@@ -109,4 +109,13 @@ typedef int (*nghttp2_pq_item_cb)(void *item, void *arg);
  */
 void nghttp2_pq_update(nghttp2_pq *pq, nghttp2_pq_item_cb fun, void *arg);
 
+/*
+ * Applys |fun| to each item in |pq|.  The |arg| is passed as arg
+ * parameter to callback function.  This function must not change the
+ * ordering key.  If the return value from callback is nonzero, this
+ * function returns 1 immediately without iterating remaining items.
+ * Otherwise this function returns 0.
+ */
+int nghttp2_pq_each(nghttp2_pq *pq, nghttp2_pq_item_cb fun, void *arg);
+
 #endif /* NGHTTP2_PQ_H */

lib/nghttp2_session.c

@@ -33,6 +33,7 @@
 #include "nghttp2_net.h"
 #include "nghttp2_priority_spec.h"
 #include "nghttp2_option.h"
+#include "nghttp2_http.h"
 
 /*
  * Returns non-zero if the number of outgoing opened streams is larger
@@ -76,6 +77,27 @@ static int is_non_fatal(int lib_error) {
 
 int nghttp2_is_fatal(int lib_error) { return lib_error < NGHTTP2_ERR_FATAL; }
 
+static int session_enforce_http_messaging(nghttp2_session *session) {
+  return (session->opt_flags & NGHTTP2_OPTMASK_NO_HTTP_MESSAGING) == 0;
+}
+
+/*
+ * Returns nonzero if |frame| is trailer headers.
+ */
+static int session_trailer_headers(nghttp2_session *session,
+                                   nghttp2_stream *stream,
+                                   nghttp2_frame *frame) {
+  if (!stream || frame->hd.type != NGHTTP2_HEADERS) {
+    return 0;
+  }
+  if (session->server) {
+    return frame->headers.cat == NGHTTP2_HCAT_HEADERS;
+  }
+
+  return frame->headers.cat == NGHTTP2_HCAT_HEADERS &&
+         (stream->http_flags & NGHTTP2_HTTP_FLAG_EXPECT_FINAL_RESPONSE) == 0;
+}
+
 /* Returns nonzero if the |stream| is in reserved(remote) state */
 static int state_reserved_remote(nghttp2_session *session,
                                  nghttp2_stream *stream) {
@@ -392,10 +414,17 @@ static int session_new(nghttp2_session **session_ptr,
           option->peer_max_concurrent_streams;
     }
 
-    if (option->opt_set_mask & NGHTTP2_OPT_RECV_CLIENT_PREFACE) {
+    if ((option->opt_set_mask & NGHTTP2_OPT_RECV_CLIENT_PREFACE) &&
+        option->recv_client_preface) {
 
       (*session_ptr)->opt_flags |= NGHTTP2_OPTMASK_RECV_CLIENT_PREFACE;
     }
+
+    if ((option->opt_set_mask & NGHTTP2_OPT_NO_HTTP_MESSAGING) &&
+        option->no_http_messaging) {
+
+      (*session_ptr)->opt_flags |= NGHTTP2_OPTMASK_NO_HTTP_MESSAGING;
+    }
   }
 
   (*session_ptr)->callbacks = *callbacks;
@@ -688,16 +717,7 @@ int nghttp2_session_add_item(nghttp2_session *session,
     switch (frame->hd.type) {
     case NGHTTP2_RST_STREAM:
       if (stream) {
-        /* We rely on the stream state to decide whether number of
-           streams should be decremented or not. For purly reserved or
-           idle streams, they are not counted to those numbers and we
-           must keep this state in order not to decrement the number.
-           We don't check against NGHTTP2_STREAM_IDLE because
-           nghttp2_session_get_stream() does not return such
-           stream. */
-        if (stream->state != NGHTTP2_STREAM_RESERVED) {
-          stream->state = NGHTTP2_STREAM_CLOSING;
-        }
+        stream->state = NGHTTP2_STREAM_CLOSING;
       }
 
       break;
@@ -781,6 +801,30 @@ int nghttp2_session_add_item(nghttp2_session *session,
   return 0;
 }
 
+typedef struct {
+  int32_t stream_id;
+  uint32_t error_code;
+} nghttp2_rst_target;
+
+static int cancel_pending_request(void *pq_item, void *arg) {
+  nghttp2_outbound_item *item;
+  nghttp2_rst_target *t;
+  nghttp2_headers_aux_data *aux_data;
+
+  item = pq_item;
+  t = arg;
+  aux_data = &item->aux_data.headers;
+
+  if (item->frame.hd.stream_id != t->stream_id || aux_data->canceled) {
+    return 0;
+  }
+
+  aux_data->error_code = t->error_code;
+  aux_data->canceled = 1;
+
+  return 1;
+}
+
 int nghttp2_session_add_rst_stream(nghttp2_session *session, int32_t stream_id,
                                    uint32_t error_code) {
   int rv;
@@ -788,6 +832,7 @@ int nghttp2_session_add_rst_stream(nghttp2_session *session, int32_t stream_id,
   nghttp2_frame *frame;
   nghttp2_stream *stream;
   nghttp2_mem *mem;
+  nghttp2_rst_target t = {stream_id, error_code};
 
   mem = &session->mem;
   stream = nghttp2_session_get_stream(session, stream_id);
@@ -795,6 +840,26 @@ int nghttp2_session_add_rst_stream(nghttp2_session *session, int32_t stream_id,
     return 0;
   }
 
+  /* Cancel pending request HEADERS in ob_ss_pq if this RST_STREAM
+     refers to that stream. */
+  if (!session->server && nghttp2_session_is_my_stream_id(session, stream_id) &&
+      nghttp2_pq_top(&session->ob_ss_pq)) {
+    nghttp2_outbound_item *top;
+    nghttp2_frame *headers_frame;
+
+    top = nghttp2_pq_top(&session->ob_ss_pq);
+    headers_frame = &top->frame;
+
+    assert(headers_frame->hd.type == NGHTTP2_HEADERS);
+
+    if (headers_frame->hd.stream_id <= stream_id &&
+        (uint32_t)stream_id < session->next_stream_id) {
+      if (nghttp2_pq_each(&session->ob_ss_pq, cancel_pending_request, &t)) {
+        return 0;
+      }
+    }
+  }
+
   item = nghttp2_mem_malloc(mem, sizeof(nghttp2_outbound_item));
   if (item == NULL) {
     return NGHTTP2_ERR_NOMEM;
@@ -879,6 +944,10 @@ nghttp2_stream *nghttp2_session_open_stream(nghttp2_session *session,
     }
   }
 
+  if (initial_state == NGHTTP2_STREAM_RESERVED) {
+    flags |= NGHTTP2_STREAM_FLAG_PUSH;
+  }
+
   nghttp2_stream_init(
       stream, stream_id, flags, initial_state, pri_spec->weight,
       &session->roots, session->remote_settings.initial_window_size,
@@ -1016,10 +1085,9 @@ int nghttp2_session_close_stream(nghttp2_session *session, int32_t stream_id,
     }
   }
 
-  switch (stream->state) {
-  case NGHTTP2_STREAM_RESERVED:
-    break;
-  default:
+  /* pushed streams which is not opened yet is not counted toward max
+     concurrent limits */
+  if ((stream->flags & NGHTTP2_STREAM_FLAG_PUSH) == 0) {
     if (nghttp2_session_is_my_stream_id(session, stream_id)) {
       --session->num_outgoing_streams;
     } else {
@@ -1249,7 +1317,7 @@ static int session_predicate_for_stream_send(nghttp2_session *session,
 }
 
 /*
- * This function checks HEADERS frame |frame|, which opens stream, can
+ * This function checks request HEADERS frame, which opens stream, can
  * be sent at this time.
  *
  * This function returns 0 if it succeeds, or one of the following
@@ -1259,8 +1327,14 @@ static int session_predicate_for_stream_send(nghttp2_session *session,
  *     New stream cannot be created because of GOAWAY: session is
  *     going down or received last_stream_id is strictly less than
  *     frame->hd.stream_id.
+ * NGHTTP2_ERR_STREAM_CLOSING
+ *     request HEADERS was canceled by RST_STREAM while it is in queue.
  */
-static int session_predicate_request_headers_send(nghttp2_session *session) {
+static int session_predicate_request_headers_send(nghttp2_session *session,
+                                                  nghttp2_outbound_item *item) {
+  if (item->aux_data.headers.canceled) {
+    return NGHTTP2_ERR_STREAM_CLOSING;
+  }
   /* If we are terminating session (NGHTTP2_GOAWAY_TERM_ON_SEND) or
      GOAWAY was received from peer, new request is not allowed. */
   if (session->goaway_flags &
@@ -1653,7 +1727,6 @@ static size_t session_estimate_headers_payload(nghttp2_session *session,
  */
 static int session_prep_frame(nghttp2_session *session,
                               nghttp2_outbound_item *item) {
-  int framerv = 0;
   int rv;
   nghttp2_frame *frame;
   nghttp2_mem *mem;
@@ -1678,13 +1751,8 @@ static int session_prep_frame(nghttp2_session *session,
       }
 
       if (frame->headers.cat == NGHTTP2_HCAT_REQUEST) {
-        nghttp2_stream *stream;
-
         /* initial HEADERS, which opens stream */
-        rv = session_predicate_request_headers_send(session);
-        if (rv != 0) {
-          return rv;
-        }
+        nghttp2_stream *stream;
 
         stream = nghttp2_session_open_stream(
             session, frame->hd.stream_id, NGHTTP2_STREAM_FLAG_NONE,
@@ -1695,6 +1763,14 @@ static int session_prep_frame(nghttp2_session *session,
           return NGHTTP2_ERR_NOMEM;
         }
 
+        rv = session_predicate_request_headers_send(session, item);
+        if (rv != 0) {
+          return rv;
+        }
+
+        if (session_enforce_http_messaging(session)) {
+          nghttp2_http_record_request_method(stream, frame);
+        }
       } else {
         nghttp2_stream *stream;
 
@@ -1731,41 +1807,27 @@ static int session_prep_frame(nghttp2_session *session,
         }
       }
 
-      framerv = nghttp2_frame_pack_headers(
-          &session->aob.framebufs, &frame->headers, &session->hd_deflater);
+      rv = nghttp2_frame_pack_headers(&session->aob.framebufs, &frame->headers,
+                                      &session->hd_deflater);
 
-      if (framerv < 0) {
-        goto close_stream_return;
+      if (rv != 0) {
+        return rv;
       }
 
       DEBUGF(fprintf(stderr,
                      "send: before padding, HEADERS serialized in %zd bytes\n",
                      nghttp2_bufs_len(&session->aob.framebufs)));
 
-      framerv = session_headers_add_pad(session, frame);
+      rv = session_headers_add_pad(session, frame);
 
-      if (framerv < 0) {
-        goto close_stream_return;
+      if (rv != 0) {
+        return rv;
       }
 
       DEBUGF(fprintf(stderr, "send: HEADERS finally serialized in %zd bytes\n",
                      nghttp2_bufs_len(&session->aob.framebufs)));
 
       break;
-
-    close_stream_return:
-
-      if (frame->headers.cat == NGHTTP2_HCAT_REQUEST &&
-          !nghttp2_is_fatal(framerv)) {
-        rv = nghttp2_session_close_stream(session, frame->hd.stream_id,
-                                          NGHTTP2_NO_ERROR);
-
-        if (nghttp2_is_fatal(rv)) {
-          return rv;
-        }
-      }
-
-      return framerv;
     }
     case NGHTTP2_PRIORITY: {
       if (session_is_closing(session)) {
@@ -1773,11 +1835,7 @@ static int session_prep_frame(nghttp2_session *session,
       }
       /* PRIORITY frame can be sent at any time and to any stream
          ID. */
-      framerv = nghttp2_frame_pack_priority(&session->aob.framebufs,
-                                            &frame->priority);
-      if (framerv < 0) {
-        return framerv;
-      }
+      nghttp2_frame_pack_priority(&session->aob.framebufs, &frame->priority);
 
       /* Peer can send PRIORITY frame against idle stream to create
          "anchor" in dependency tree.  Only client can do this in
@@ -1789,17 +1847,14 @@ static int session_prep_frame(nghttp2_session *session,
       if (session_is_closing(session)) {
         return NGHTTP2_ERR_SESSION_CLOSING;
       }
-      framerv = nghttp2_frame_pack_rst_stream(&session->aob.framebufs,
-                                              &frame->rst_stream);
-      if (framerv < 0) {
-        return framerv;
-      }
+      nghttp2_frame_pack_rst_stream(&session->aob.framebufs,
+                                    &frame->rst_stream);
       break;
     case NGHTTP2_SETTINGS: {
-      framerv = nghttp2_frame_pack_settings(&session->aob.framebufs,
-                                            &frame->settings);
-      if (framerv < 0) {
-        return framerv;
+      rv = nghttp2_frame_pack_settings(&session->aob.framebufs,
+                                       &frame->settings);
+      if (rv != 0) {
+        return rv;
       }
       break;
     }
@@ -1811,6 +1866,24 @@ static int session_prep_frame(nghttp2_session *session,
 
       aux_data = &item->aux_data.headers;
 
+      stream = nghttp2_session_get_stream(session, frame->hd.stream_id);
+
+      /* stream could be NULL if associated stream was already
+         closed. */
+      if (stream) {
+        nghttp2_priority_spec_init(&pri_spec, stream->stream_id,
+                                   NGHTTP2_DEFAULT_WEIGHT, 0);
+      } else {
+        nghttp2_priority_spec_default_init(&pri_spec);
+      }
+
+      if (!nghttp2_session_open_stream(
+              session, frame->push_promise.promised_stream_id,
+              NGHTTP2_STREAM_FLAG_NONE, &pri_spec, NGHTTP2_STREAM_RESERVED,
+              aux_data->stream_user_data)) {
+        return NGHTTP2_ERR_NOMEM;
+      }
+
       estimated_payloadlen = session_estimate_headers_payload(
           session, frame->push_promise.nva, frame->push_promise.nvlen, 0);
 
@@ -1818,8 +1891,7 @@ static int session_prep_frame(nghttp2_session *session,
         return NGHTTP2_ERR_FRAME_SIZE_ERROR;
       }
 
-      stream = nghttp2_session_get_stream(session, frame->hd.stream_id);
-
+      /* predicte should fail if stream is NULL. */
       rv = session_predicate_push_promise_send(session, stream);
       if (rv != 0) {
         return rv;
@@ -1827,55 +1899,37 @@ static int session_prep_frame(nghttp2_session *session,
 
       assert(stream);
 
-      framerv = nghttp2_frame_pack_push_promise(
+      rv = nghttp2_frame_pack_push_promise(
           &session->aob.framebufs, &frame->push_promise, &session->hd_deflater);
-      if (framerv < 0) {
-        return framerv;
+      if (rv != 0) {
+        return rv;
       }
-      framerv = session_headers_add_pad(session, frame);
-      if (framerv < 0) {
-        return framerv;
+      rv = session_headers_add_pad(session, frame);
+      if (rv != 0) {
+        return rv;
       }
 
-      /* TODO It is unclear reserved stream dpeneds on associated
-         stream with or without exclusive flag set */
-      nghttp2_priority_spec_init(&pri_spec, stream->stream_id,
-                                 NGHTTP2_DEFAULT_WEIGHT, 0);
-
-      if (!nghttp2_session_open_stream(
-              session, frame->push_promise.promised_stream_id,
-              NGHTTP2_STREAM_FLAG_PUSH, &pri_spec, NGHTTP2_STREAM_RESERVED,
-              aux_data->stream_user_data)) {
-        return NGHTTP2_ERR_NOMEM;
-      }
       break;
     }
     case NGHTTP2_PING:
       if (session_is_closing(session)) {
         return NGHTTP2_ERR_SESSION_CLOSING;
       }
-      framerv = nghttp2_frame_pack_ping(&session->aob.framebufs, &frame->ping);
-      if (framerv < 0) {
-        return framerv;
-      }
+      nghttp2_frame_pack_ping(&session->aob.framebufs, &frame->ping);
       break;
     case NGHTTP2_WINDOW_UPDATE: {
       rv = session_predicate_window_update_send(session, frame->hd.stream_id);
       if (rv != 0) {
         return rv;
       }
-      framerv = nghttp2_frame_pack_window_update(&session->aob.framebufs,
-                                                 &frame->window_update);
-      if (framerv < 0) {
-        return framerv;
-      }
+      nghttp2_frame_pack_window_update(&session->aob.framebufs,
+                                       &frame->window_update);
       break;
     }
     case NGHTTP2_GOAWAY:
-      framerv =
-          nghttp2_frame_pack_goaway(&session->aob.framebufs, &frame->goaway);
-      if (framerv < 0) {
-        return framerv;
+      rv = nghttp2_frame_pack_goaway(&session->aob.framebufs, &frame->goaway);
+      if (rv != 0) {
+        return rv;
       }
       session->local_last_stream_id = frame->goaway.last_stream_id;
 
@@ -1930,10 +1984,9 @@ static int session_prep_frame(nghttp2_session *session,
       return NGHTTP2_ERR_DEFERRED;
     }
 
-    framerv =
-        nghttp2_session_pack_data(session, &session->aob.framebufs,
-                                  next_readmax, frame, &item->aux_data.data);
-    if (framerv == NGHTTP2_ERR_DEFERRED) {
+    rv = nghttp2_session_pack_data(session, &session->aob.framebufs,
+                                   next_readmax, frame, &item->aux_data.data);
+    if (rv == NGHTTP2_ERR_DEFERRED) {
       rv = nghttp2_stream_defer_item(stream, NGHTTP2_STREAM_FLAG_DEFERRED_USER,
                                      session);
 
@@ -1945,7 +1998,7 @@ static int session_prep_frame(nghttp2_session *session,
       active_outbound_item_reset(&session->aob, mem);
       return NGHTTP2_ERR_DEFERRED;
     }
-    if (framerv == NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE) {
+    if (rv == NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE) {
       rv = nghttp2_stream_detach_item(stream, session);
 
       if (nghttp2_is_fatal(rv)) {
@@ -1954,19 +2007,21 @@ static int session_prep_frame(nghttp2_session *session,
 
       rv = nghttp2_session_add_rst_stream(session, frame->hd.stream_id,
                                           NGHTTP2_INTERNAL_ERROR);
-      if (rv != 0) {
-        return rv;
-      }
-      return framerv;
-    }
-    if (framerv < 0) {
-      rv = nghttp2_stream_detach_item(stream, session);
-
       if (nghttp2_is_fatal(rv)) {
         return rv;
       }
+      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
+    }
+    if (rv != 0) {
+      int rv2;
 
-      return framerv;
+      rv2 = nghttp2_stream_detach_item(stream, session);
+
+      if (nghttp2_is_fatal(rv2)) {
+        return rv2;
+      }
+
+      return rv;
     }
     return 0;
   }
@@ -2290,6 +2345,7 @@ static int session_after_frame_sent1(nghttp2_session *session) {
         break;
       }
       case NGHTTP2_HCAT_PUSH_RESPONSE:
+        stream->flags &= ~NGHTTP2_STREAM_FLAG_PUSH;
         ++session->num_outgoing_streams;
       /* Fall through */
       case NGHTTP2_HCAT_RESPONSE:
@@ -2523,8 +2579,14 @@ static int session_after_frame_sent2(nghttp2_session *session) {
     assert(stream);
     next_item = nghttp2_session_get_next_ob_item(session);
 
-    session_outbound_item_cycle_weight(session, aob->item,
-                                       stream->effective_weight);
+    /* Imagine we hit connection window size limit while sending DATA
+       frame.  If we decrement weight here, its stream might get
+       inferior share because the other streams' weight is not
+       decremented because of flow control. */
+    if (session->remote_window_size > 0 || stream->remote_window_size <= 0) {
+      session_outbound_item_cycle_weight(session, aob->item,
+                                         stream->effective_weight);
+    }
 
     /* If priority of this stream is higher or equal to other stream
        waiting at the top of the queue, we continue to send this
@@ -2675,6 +2737,9 @@ static ssize_t nghttp2_session_mem_send_internal(nghttp2_session *session,
         break;
       }
       if (rv < 0) {
+        int32_t opened_stream_id = 0;
+        uint32_t error_code = NGHTTP2_INTERNAL_ERROR;
+
         DEBUGF(fprintf(stderr, "send: frame preparation failed with %s\n",
                        nghttp2_strerror(rv)));
         /* TODO If the error comes from compressor, the connection
@@ -2685,17 +2750,42 @@ static ssize_t nghttp2_session_mem_send_internal(nghttp2_session *session,
           /* The library is responsible for the transmission of
              WINDOW_UPDATE frame, so we don't call error callback for
              it. */
-          if (frame->hd.type != NGHTTP2_WINDOW_UPDATE) {
-            if (session->callbacks.on_frame_not_send_callback(
-                    session, frame, rv, session->user_data) != 0) {
+          if (frame->hd.type != NGHTTP2_WINDOW_UPDATE &&
+              session->callbacks.on_frame_not_send_callback(
+                  session, frame, rv, session->user_data) != 0) {
 
-              nghttp2_outbound_item_free(item, mem);
-              nghttp2_mem_free(mem, item);
+            nghttp2_outbound_item_free(item, mem);
+            nghttp2_mem_free(mem, item);
 
-              return NGHTTP2_ERR_CALLBACK_FAILURE;
-            }
+            return NGHTTP2_ERR_CALLBACK_FAILURE;
           }
         }
+        /* We have to close stream opened by failed request HEADERS
+           or PUSH_PROMISE. */
+        switch (item->frame.hd.type) {
+        case NGHTTP2_HEADERS:
+          if (item->frame.headers.cat == NGHTTP2_HCAT_REQUEST) {
+            opened_stream_id = item->frame.hd.stream_id;
+            if (item->aux_data.headers.canceled) {
+              error_code = item->aux_data.headers.error_code;
+            }
+          }
+          break;
+        case NGHTTP2_PUSH_PROMISE:
+          opened_stream_id = item->frame.push_promise.promised_stream_id;
+          break;
+        }
+        if (opened_stream_id) {
+          /* careful not to override rv */
+          int rv2;
+          rv2 = nghttp2_session_close_stream(session, opened_stream_id,
+                                             error_code);
+
+          if (nghttp2_is_fatal(rv2)) {
+            return rv2;
+          }
+        }
+
         nghttp2_outbound_item_free(item, mem);
         nghttp2_mem_free(mem, item);
         active_outbound_item_reset(aob, mem);
@@ -2736,7 +2826,7 @@ static ssize_t nghttp2_session_mem_send_internal(nghttp2_session *session,
 
       DEBUGF(fprintf(stderr,
                      "send: start transmitting frame type=%u, length=%zd\n",
-                     framebufs->cur->buf.pos[2],
+                     framebufs->cur->buf.pos[3],
                      framebufs->cur->buf.last - framebufs->cur->buf.pos));
 
       aob->state = NGHTTP2_OB_SEND_DATA;
@@ -2933,11 +3023,12 @@ static int session_handle_frame_size_error(nghttp2_session *session,
   return nghttp2_session_terminate_session(session, NGHTTP2_FRAME_SIZE_ERROR);
 }
 
-static int session_handle_invalid_stream(nghttp2_session *session,
-                                         nghttp2_frame *frame,
-                                         uint32_t error_code) {
+static int session_handle_invalid_stream2(nghttp2_session *session,
+                                          int32_t stream_id,
+                                          nghttp2_frame *frame,
+                                          uint32_t error_code) {
   int rv;
-  rv = nghttp2_session_add_rst_stream(session, frame->hd.stream_id, error_code);
+  rv = nghttp2_session_add_rst_stream(session, stream_id, error_code);
   if (rv != 0) {
     return rv;
   }
@@ -2950,6 +3041,13 @@ static int session_handle_invalid_stream(nghttp2_session *session,
   return 0;
 }
 
+static int session_handle_invalid_stream(nghttp2_session *session,
+                                         nghttp2_frame *frame,
+                                         uint32_t error_code) {
+  return session_handle_invalid_stream2(session, frame->hd.stream_id, frame,
+                                        error_code);
+}
+
 static int session_inflate_handle_invalid_stream(nghttp2_session *session,
                                                  nghttp2_frame *frame,
                                                  uint32_t error_code) {
@@ -3025,8 +3123,19 @@ static int inflate_header_block(nghttp2_session *session, nghttp2_frame *frame,
   int inflate_flags;
   nghttp2_nv nv;
   nghttp2_stream *stream;
+  nghttp2_stream *subject_stream;
+  int trailer = 0;
 
   *readlen_ptr = 0;
+  stream = nghttp2_session_get_stream(session, frame->hd.stream_id);
+
+  if (frame->hd.type == NGHTTP2_PUSH_PROMISE) {
+    subject_stream = nghttp2_session_get_stream(
+        session, frame->push_promise.promised_stream_id);
+  } else {
+    subject_stream = stream;
+    trailer = session_trailer_headers(session, stream, frame);
+  }
 
   DEBUGF(fprintf(stderr, "recv: decoding header block %zu bytes\n", inlen));
   for (;;) {
@@ -3038,8 +3147,6 @@ static int inflate_header_block(nghttp2_session *session, nghttp2_frame *frame,
     }
     if (proclen < 0) {
       if (session->iframe.state == NGHTTP2_IB_READ_HEADER_BLOCK) {
-        stream = nghttp2_session_get_stream(session, frame->hd.stream_id);
-
         if (stream && stream->state != NGHTTP2_STREAM_CLOSING) {
           /* Adding RST_STREAM here is very important. It prevents
              from invoking subsequent callbacks for the same stream
@@ -3067,11 +3174,40 @@ static int inflate_header_block(nghttp2_session *session, nghttp2_frame *frame,
     DEBUGF(fprintf(stderr, "recv: proclen=%zd\n", proclen));
 
     if (call_header_cb && (inflate_flags & NGHTTP2_HD_INFLATE_EMIT)) {
-      rv = session_call_on_header(session, frame, &nv);
-      /* This handles NGHTTP2_ERR_PAUSE and
-         NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE as well */
-      if (rv != 0) {
-        return rv;
+      rv = 0;
+      if (subject_stream && session_enforce_http_messaging(session)) {
+        rv = nghttp2_http_on_header(session, subject_stream, frame, &nv,
+                                    trailer);
+        if (rv == NGHTTP2_ERR_HTTP_HEADER) {
+          DEBUGF(fprintf(
+              stderr, "recv: HTTP error: type=%d, id=%d, header %.*s: %.*s\n",
+              frame->hd.type, subject_stream->stream_id, (int)nv.namelen,
+              nv.name, (int)nv.valuelen, nv.value));
+
+          rv =
+              session_handle_invalid_stream2(session, subject_stream->stream_id,
+                                             frame, NGHTTP2_PROTOCOL_ERROR);
+          if (nghttp2_is_fatal(rv)) {
+            return rv;
+          }
+          return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
+        }
+
+        if (rv == NGHTTP2_ERR_IGN_HTTP_HEADER) {
+          /* header is ignored */
+          DEBUGF(fprintf(
+              stderr, "recv: HTTP ignored: type=%d, id=%d, header %.*s: %.*s\n",
+              frame->hd.type, subject_stream->stream_id, (int)nv.namelen,
+              nv.name, (int)nv.valuelen, nv.value));
+        }
+      }
+      if (rv == 0) {
+        rv = session_call_on_header(session, frame, &nv);
+        /* This handles NGHTTP2_ERR_PAUSE and
+           NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE as well */
+        if (rv != 0) {
+          return rv;
+        }
       }
     }
     if (inflate_flags & NGHTTP2_HD_INFLATE_FINAL) {
@@ -3167,7 +3303,8 @@ int nghttp2_session_end_headers_received(nghttp2_session *session,
 }
 
 static int session_after_header_block_received(nghttp2_session *session) {
-  int rv;
+  int rv = 0;
+  int call_cb = 1;
   nghttp2_frame *frame = &session->iframe.frame;
   nghttp2_stream *stream;
 
@@ -3178,9 +3315,64 @@ static int session_after_header_block_received(nghttp2_session *session) {
     return 0;
   }
 
-  rv = session_call_on_frame_received(session, frame);
-  if (nghttp2_is_fatal(rv)) {
-    return rv;
+  if (session_enforce_http_messaging(session)) {
+    if (frame->hd.type == NGHTTP2_PUSH_PROMISE) {
+      nghttp2_stream *subject_stream;
+
+      subject_stream = nghttp2_session_get_stream(
+          session, frame->push_promise.promised_stream_id);
+      if (subject_stream) {
+        rv = nghttp2_http_on_request_headers(subject_stream, frame);
+      }
+    } else {
+      assert(frame->hd.type == NGHTTP2_HEADERS);
+      switch (frame->headers.cat) {
+      case NGHTTP2_HCAT_REQUEST:
+        rv = nghttp2_http_on_request_headers(stream, frame);
+        break;
+      case NGHTTP2_HCAT_RESPONSE:
+      case NGHTTP2_HCAT_PUSH_RESPONSE:
+        rv = nghttp2_http_on_response_headers(stream);
+        break;
+      case NGHTTP2_HCAT_HEADERS:
+        if (stream->http_flags & NGHTTP2_HTTP_FLAG_EXPECT_FINAL_RESPONSE) {
+          assert(!session->server);
+          rv = nghttp2_http_on_response_headers(stream);
+        } else {
+          rv = nghttp2_http_on_trailer_headers(stream, frame);
+        }
+        break;
+      default:
+        assert(0);
+      }
+      if (rv == 0 && (frame->hd.flags & NGHTTP2_FLAG_END_STREAM)) {
+        rv = nghttp2_http_on_remote_end_stream(stream);
+      }
+    }
+    if (rv != 0) {
+      int32_t stream_id;
+
+      if (frame->hd.type == NGHTTP2_PUSH_PROMISE) {
+        stream_id = frame->push_promise.promised_stream_id;
+      } else {
+        stream_id = frame->hd.stream_id;
+      }
+
+      call_cb = 0;
+
+      rv = session_handle_invalid_stream2(session, stream_id, frame,
+                                          NGHTTP2_PROTOCOL_ERROR);
+      if (nghttp2_is_fatal(rv)) {
+        return rv;
+      }
+    }
+  }
+
+  if (call_cb) {
+    rv = session_call_on_frame_received(session, frame);
+    if (nghttp2_is_fatal(rv)) {
+      return rv;
+    }
   }
 
   if (frame->hd.type != NGHTTP2_HEADERS) {
@@ -3250,7 +3442,7 @@ int nghttp2_session_on_request_headers_received(nghttp2_session *session,
 
   if (session_is_incoming_concurrent_streams_max(session)) {
     return session_inflate_handle_invalid_connection(
-        session, frame, NGHTTP2_ENHANCE_YOUR_CALM,
+        session, frame, NGHTTP2_PROTOCOL_ERROR,
         "request HEADERS: max concurrent streams exceeded");
   }
 
@@ -3327,7 +3519,7 @@ int nghttp2_session_on_push_response_headers_received(nghttp2_session *session,
 
   if (session_is_incoming_concurrent_streams_max(session)) {
     return session_inflate_handle_invalid_connection(
-        session, frame, NGHTTP2_ENHANCE_YOUR_CALM,
+        session, frame, NGHTTP2_PROTOCOL_ERROR,
         "push response HEADERS: max concurrent streams exceeded");
   }
   if (session_is_incoming_concurrent_streams_pending_max(session)) {
@@ -3976,7 +4168,7 @@ int nghttp2_session_on_push_promise_received(nghttp2_session *session,
                              NGHTTP2_DEFAULT_WEIGHT, 0);
 
   promised_stream = nghttp2_session_open_stream(
-      session, frame->push_promise.promised_stream_id, NGHTTP2_STREAM_FLAG_PUSH,
+      session, frame->push_promise.promised_stream_id, NGHTTP2_STREAM_FLAG_NONE,
       &pri_spec, NGHTTP2_STREAM_RESERVED, NULL);
 
   if (!promised_stream) {
@@ -4177,6 +4369,7 @@ static int session_process_window_update_frame(nghttp2_session *session) {
 int nghttp2_session_on_data_received(nghttp2_session *session,
                                      nghttp2_frame *frame) {
   int rv = 0;
+  int call_cb = 1;
   nghttp2_stream *stream;
 
   /* We don't call on_frame_recv_callback if stream has been closed
@@ -4189,9 +4382,23 @@ int nghttp2_session_on_data_received(nghttp2_session *session,
     return 0;
   }
 
-  rv = session_call_on_frame_received(session, frame);
-  if (nghttp2_is_fatal(rv)) {
-    return rv;
+  if (session_enforce_http_messaging(session) &&
+      (frame->hd.flags & NGHTTP2_FLAG_END_STREAM)) {
+    if (nghttp2_http_on_remote_end_stream(stream) != 0) {
+      call_cb = 0;
+      rv = nghttp2_session_add_rst_stream(session, stream->stream_id,
+                                          NGHTTP2_PROTOCOL_ERROR);
+      if (nghttp2_is_fatal(rv)) {
+        return rv;
+      }
+    }
+  }
+
+  if (call_cb) {
+    rv = session_call_on_frame_received(session, frame);
+    if (nghttp2_is_fatal(rv)) {
+      return rv;
+    }
   }
 
   if (frame->hd.flags & NGHTTP2_FLAG_END_STREAM) {
@@ -4625,13 +4832,10 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
         return in - first;
       }
 
-      if (iframe->sbuf.pos[3] != NGHTTP2_SETTINGS) {
-        nghttp2_frame_unpack_frame_hd(&iframe->frame.hd, iframe->sbuf.pos);
-        iframe->payloadleft = iframe->frame.hd.length;
+      if (iframe->sbuf.pos[3] != NGHTTP2_SETTINGS ||
+          (iframe->sbuf.pos[4] & NGHTTP2_FLAG_ACK)) {
 
-        busy = 1;
-
-        iframe->state = NGHTTP2_IB_IGN_PAYLOAD;
+        iframe->state = NGHTTP2_IB_IGN_ALL;
 
         rv = nghttp2_session_terminate_session_with_reason(
             session, NGHTTP2_PROTOCOL_ERROR, "SETTINGS expected");
@@ -4640,7 +4844,7 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
           return rv;
         }
 
-        break;
+        return inlen;
       }
 
       iframe->state = NGHTTP2_IB_READ_HEAD;
@@ -5528,17 +5732,30 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
 
         DEBUGF(fprintf(stderr, "recv: data_readlen=%zd\n", data_readlen));
 
-        if (stream && data_readlen > 0 &&
-            session->callbacks.on_data_chunk_recv_callback) {
-          rv = session->callbacks.on_data_chunk_recv_callback(
-              session, iframe->frame.hd.flags, iframe->frame.hd.stream_id,
-              in - readlen, data_readlen, session->user_data);
-          if (rv == NGHTTP2_ERR_PAUSE) {
-            return in - first;
+        if (stream && data_readlen > 0) {
+          if (session_enforce_http_messaging(session)) {
+            if (nghttp2_http_on_data_chunk(stream, data_readlen) != 0) {
+              rv = nghttp2_session_add_rst_stream(
+                  session, iframe->frame.hd.stream_id, NGHTTP2_PROTOCOL_ERROR);
+              if (nghttp2_is_fatal(rv)) {
+                return rv;
+              }
+              busy = 1;
+              iframe->state = NGHTTP2_IB_IGN_DATA;
+              break;
+            }
           }
+          if (session->callbacks.on_data_chunk_recv_callback) {
+            rv = session->callbacks.on_data_chunk_recv_callback(
+                session, iframe->frame.hd.flags, iframe->frame.hd.stream_id,
+                in - readlen, data_readlen, session->user_data);
+            if (rv == NGHTTP2_ERR_PAUSE) {
+              return in - first;
+            }
 
-          if (nghttp2_is_fatal(rv)) {
-            return NGHTTP2_ERR_CALLBACK_FAILURE;
+            if (nghttp2_is_fatal(rv)) {
+              return NGHTTP2_ERR_CALLBACK_FAILURE;
+            }
           }
         }
       }
@@ -5591,6 +5808,8 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
       session_inbound_frame_reset(session);
 
       break;
+    case NGHTTP2_IB_IGN_ALL:
+      return inlen;
     }
 
     if (!busy && in == last) {

lib/nghttp2_session.h

@@ -47,6 +47,7 @@
 typedef enum {
   NGHTTP2_OPTMASK_NO_AUTO_WINDOW_UPDATE = 1 << 0,
   NGHTTP2_OPTMASK_RECV_CLIENT_PREFACE = 1 << 1,
+  NGHTTP2_OPTMASK_NO_HTTP_MESSAGING = 1 << 2,
 } nghttp2_optmask;
 
 typedef enum {
@@ -81,7 +82,8 @@ typedef enum {
   NGHTTP2_IB_IGN_CONTINUATION,
   NGHTTP2_IB_READ_PAD_DATA,
   NGHTTP2_IB_READ_DATA,
-  NGHTTP2_IB_IGN_DATA
+  NGHTTP2_IB_IGN_DATA,
+  NGHTTP2_IB_IGN_ALL,
 } nghttp2_inbound_state;
 
 #define NGHTTP2_INBOUND_NUM_IV 7
@@ -400,6 +402,9 @@ int nghttp2_session_add_settings(nghttp2_session *session, uint8_t flags,
  * is a pointer to the arbitrary user supplied data to be associated
  * to this stream.
  *
+ * If |initial_state| is NGHTTP2_STREAM_RESERVED, this function sets
+ * NGHTTP2_STREAM_FLAG_PUSH flag set.
+ *
  * This function returns a pointer to created new stream object, or
  * NULL.
  */

lib/nghttp2_stream.c

@@ -68,6 +68,11 @@ void nghttp2_stream_init(nghttp2_stream *stream, int32_t stream_id,
   stream->roots = roots;
   stream->root_prev = NULL;
   stream->root_next = NULL;
+
+  stream->http_flags = NGHTTP2_HTTP_FLAG_NONE;
+  stream->content_length = -1;
+  stream->recv_content_length = 0;
+  stream->status_code = -1;
 }
 
 void nghttp2_stream_free(nghttp2_stream *stream _U_) {
@@ -515,6 +520,7 @@ int nghttp2_stream_update_local_initial_window_size(
 
 void nghttp2_stream_promise_fulfilled(nghttp2_stream *stream) {
   stream->state = NGHTTP2_STREAM_OPENED;
+  stream->flags &= ~NGHTTP2_STREAM_FLAG_PUSH;
 }
 
 nghttp2_stream *nghttp2_stream_get_dep_root(nghttp2_stream *stream) {

lib/nghttp2_stream.h

@@ -84,7 +84,8 @@ typedef enum {
 
 typedef enum {
   NGHTTP2_STREAM_FLAG_NONE = 0,
-  /* Indicates that this stream is pushed stream */
+  /* Indicates that this stream is pushed stream and not opened
+     yet. */
   NGHTTP2_STREAM_FLAG_PUSH = 0x01,
   /* Indicates that this stream was closed */
   NGHTTP2_STREAM_FLAG_CLOSED = 0x02,
@@ -98,6 +99,33 @@ typedef enum {
 
 } nghttp2_stream_flag;
 
+/* HTTP related flags to enforce HTTP semantics */
+typedef enum {
+  NGHTTP2_HTTP_FLAG_NONE = 0,
+  /* header field seen so far */
+  NGHTTP2_HTTP_FLAG__AUTHORITY = 1,
+  NGHTTP2_HTTP_FLAG__PATH = 1 << 1,
+  NGHTTP2_HTTP_FLAG__METHOD = 1 << 2,
+  NGHTTP2_HTTP_FLAG__SCHEME = 1 << 3,
+  /* host is not pseudo header, but we require either host or
+     :authority */
+  NGHTTP2_HTTP_FLAG_HOST = 1 << 4,
+  NGHTTP2_HTTP_FLAG__STATUS = 1 << 5,
+  /* required header fields for HTTP request except for CONNECT
+     method. */
+  NGHTTP2_HTTP_FLAG_REQ_HEADERS = NGHTTP2_HTTP_FLAG__METHOD |
+                                  NGHTTP2_HTTP_FLAG__PATH |
+                                  NGHTTP2_HTTP_FLAG__SCHEME,
+  NGHTTP2_HTTP_FLAG_PSEUDO_HEADER_DISALLOWED = 1 << 6,
+  /* HTTP method flags */
+  NGHTTP2_HTTP_FLAG_METH_CONNECT = 1 << 7,
+  NGHTTP2_HTTP_FLAG_METH_HEAD = 1 << 8,
+  NGHTTP2_HTTP_FLAG_METH_ALL =
+      NGHTTP2_HTTP_FLAG_METH_CONNECT | NGHTTP2_HTTP_FLAG_METH_HEAD,
+  /* set if final response is expected */
+  NGHTTP2_HTTP_FLAG_EXPECT_FINAL_RESPONSE = 1 << 9,
+} nghttp2_http_flag;
+
 typedef enum {
   NGHTTP2_STREAM_DPRI_NONE = 0,
   NGHTTP2_STREAM_DPRI_NO_ITEM = 0x01,
@@ -183,6 +211,14 @@ struct nghttp2_stream {
   uint8_t flags;
   /* Bitwise OR of zero or more nghttp2_shut_flag values */
   uint8_t shut_flags;
+  /* Content-Length of request/response body.  -1 if unknown. */
+  int64_t content_length;
+  /* Received body so far */
+  int64_t recv_content_length;
+  /* status code from remote server */
+  int16_t status_code;
+  /* Bitwise OR of zero or more nghttp2_http_flag values */
+  uint16_t http_flags;
 };
 
 void nghttp2_stream_init(nghttp2_stream *stream, int32_t stream_id,

m4/libxml2.m4

@@ -0,0 +1,188 @@
+# Configure paths for LIBXML2
+# Mike Hommey 2004-06-19
+# use CPPFLAGS instead of CFLAGS
+# Toshio Kuratomi 2001-04-21
+# Adapted from:
+# Configure paths for GLIB
+# Owen Taylor     97-11-3
+
+dnl AM_PATH_XML2([MINIMUM-VERSION, [ACTION-IF-FOUND [, ACTION-IF-NOT-FOUND]]])
+dnl Test for XML, and define XML_CPPFLAGS and XML_LIBS
+dnl
+AC_DEFUN([AM_PATH_XML2],[ 
+AC_ARG_WITH(xml-prefix,
+            [  --with-xml-prefix=PFX   Prefix where libxml is installed (optional)],
+            xml_config_prefix="$withval", xml_config_prefix="")
+AC_ARG_WITH(xml-exec-prefix,
+            [  --with-xml-exec-prefix=PFX Exec prefix where libxml is installed (optional)],
+            xml_config_exec_prefix="$withval", xml_config_exec_prefix="")
+AC_ARG_ENABLE(xmltest,
+              [  --disable-xmltest       Do not try to compile and run a test LIBXML program],,
+              enable_xmltest=yes)
+
+  if test x$xml_config_exec_prefix != x ; then
+     xml_config_args="$xml_config_args"
+     if test x${XML2_CONFIG+set} != xset ; then
+        XML2_CONFIG=$xml_config_exec_prefix/bin/xml2-config
+     fi
+  fi
+  if test x$xml_config_prefix != x ; then
+     xml_config_args="$xml_config_args --prefix=$xml_config_prefix"
+     if test x${XML2_CONFIG+set} != xset ; then
+        XML2_CONFIG=$xml_config_prefix/bin/xml2-config
+     fi
+  fi
+
+  AC_PATH_PROG(XML2_CONFIG, xml2-config, no)
+  min_xml_version=ifelse([$1], ,2.0.0,[$1])
+  AC_MSG_CHECKING(for libxml - version >= $min_xml_version)
+  no_xml=""
+  if test "$XML2_CONFIG" = "no" ; then
+    no_xml=yes
+  else
+    XML_CPPFLAGS=`$XML2_CONFIG $xml_config_args --cflags`
+    XML_LIBS=`$XML2_CONFIG $xml_config_args --libs`
+    xml_config_major_version=`$XML2_CONFIG $xml_config_args --version | \
+           sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\1/'`
+    xml_config_minor_version=`$XML2_CONFIG $xml_config_args --version | \
+           sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\2/'`
+    xml_config_micro_version=`$XML2_CONFIG $xml_config_args --version | \
+           sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\3/'`
+    if test "x$enable_xmltest" = "xyes" ; then
+      ac_save_CPPFLAGS="$CPPFLAGS"
+      ac_save_LIBS="$LIBS"
+      CPPFLAGS="$CPPFLAGS $XML_CPPFLAGS"
+      LIBS="$XML_LIBS $LIBS"
+dnl
+dnl Now check if the installed libxml is sufficiently new.
+dnl (Also sanity checks the results of xml2-config to some extent)
+dnl
+      rm -f conf.xmltest
+      AC_TRY_RUN([
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <libxml/xmlversion.h>
+
+int 
+main()
+{
+  int xml_major_version, xml_minor_version, xml_micro_version;
+  int major, minor, micro;
+  char *tmp_version;
+
+  system("touch conf.xmltest");
+
+  /* Capture xml2-config output via autoconf/configure variables */
+  /* HP/UX 9 (%@#!) writes to sscanf strings */
+  tmp_version = (char *)strdup("$min_xml_version");
+  if (sscanf(tmp_version, "%d.%d.%d", &major, &minor, &micro) != 3) {
+     printf("%s, bad version string from xml2-config\n", "$min_xml_version");
+     exit(1);
+   }
+   free(tmp_version);
+
+   /* Capture the version information from the header files */
+   tmp_version = (char *)strdup(LIBXML_DOTTED_VERSION);
+   if (sscanf(tmp_version, "%d.%d.%d", &xml_major_version, &xml_minor_version, &xml_micro_version) != 3) {
+     printf("%s, bad version string from libxml includes\n", "LIBXML_DOTTED_VERSION");
+     exit(1);
+   }
+   free(tmp_version);
+
+ /* Compare xml2-config output to the libxml headers */
+  if ((xml_major_version != $xml_config_major_version) ||
+      (xml_minor_version != $xml_config_minor_version) ||
+      (xml_micro_version != $xml_config_micro_version))
+    {
+      printf("*** libxml header files (version %d.%d.%d) do not match\n",
+         xml_major_version, xml_minor_version, xml_micro_version);
+      printf("*** xml2-config (version %d.%d.%d)\n",
+         $xml_config_major_version, $xml_config_minor_version, $xml_config_micro_version);
+      return 1;
+    } 
+/* Compare the headers to the library to make sure we match */
+  /* Less than ideal -- doesn't provide us with return value feedback, 
+   * only exits if there's a serious mismatch between header and library.
+   */
+    LIBXML_TEST_VERSION;
+
+    /* Test that the library is greater than our minimum version */
+    if ((xml_major_version > major) ||
+        ((xml_major_version == major) && (xml_minor_version > minor)) ||
+        ((xml_major_version == major) && (xml_minor_version == minor) &&
+        (xml_micro_version >= micro)))
+      {
+        return 0;
+       }
+     else
+      {
+        printf("\n*** An old version of libxml (%d.%d.%d) was found.\n",
+               xml_major_version, xml_minor_version, xml_micro_version);
+        printf("*** You need a version of libxml newer than %d.%d.%d. The latest version of\n",
+           major, minor, micro);
+        printf("*** libxml is always available from ftp://ftp.xmlsoft.org.\n");
+        printf("***\n");
+        printf("*** If you have already installed a sufficiently new version, this error\n");
+        printf("*** probably means that the wrong copy of the xml2-config shell script is\n");
+        printf("*** being found. The easiest way to fix this is to remove the old version\n");
+        printf("*** of LIBXML, but you can also set the XML2_CONFIG environment to point to the\n");
+        printf("*** correct copy of xml2-config. (In this case, you will have to\n");
+        printf("*** modify your LD_LIBRARY_PATH enviroment variable, or edit /etc/ld.so.conf\n");
+        printf("*** so that the correct libraries are found at run-time))\n");
+    }
+  return 1;
+}
+],, no_xml=yes,[echo $ac_n "cross compiling; assumed OK... $ac_c"])
+       CPPFLAGS="$ac_save_CPPFLAGS"
+       LIBS="$ac_save_LIBS"
+     fi
+  fi
+
+  if test "x$no_xml" = x ; then
+     AC_MSG_RESULT(yes (version $xml_config_major_version.$xml_config_minor_version.$xml_config_micro_version))
+     ifelse([$2], , :, [$2])     
+  else
+     AC_MSG_RESULT(no)
+     if test "$XML2_CONFIG" = "no" ; then
+       echo "*** The xml2-config script installed by LIBXML could not be found"
+       echo "*** If libxml was installed in PREFIX, make sure PREFIX/bin is in"
+       echo "*** your path, or set the XML2_CONFIG environment variable to the"
+       echo "*** full path to xml2-config."
+     else
+       if test -f conf.xmltest ; then
+        :
+       else
+          echo "*** Could not run libxml test program, checking why..."
+          CPPFLAGS="$CPPFLAGS $XML_CPPFLAGS"
+          LIBS="$LIBS $XML_LIBS"
+          AC_TRY_LINK([
+#include <libxml/xmlversion.h>
+#include <stdio.h>
+],      [ LIBXML_TEST_VERSION; return 0;],
+        [ echo "*** The test program compiled, but did not run. This usually means"
+          echo "*** that the run-time linker is not finding LIBXML or finding the wrong"
+          echo "*** version of LIBXML. If it is not finding LIBXML, you'll need to set your"
+          echo "*** LD_LIBRARY_PATH environment variable, or edit /etc/ld.so.conf to point"
+          echo "*** to the installed location  Also, make sure you have run ldconfig if that"
+          echo "*** is required on your system"
+          echo "***"
+          echo "*** If you have an old version installed, it is best to remove it, although"
+          echo "*** you may also be able to get things to work by modifying LD_LIBRARY_PATH" ],
+        [ echo "*** The test program failed to compile or link. See the file config.log for the"
+          echo "*** exact error that occured. This usually means LIBXML was incorrectly installed"
+          echo "*** or that you have moved LIBXML since it was installed. In the latter case, you"
+          echo "*** may want to edit the xml2-config script: $XML2_CONFIG" ])
+          CPPFLAGS="$ac_save_CPPFLAGS"
+          LIBS="$ac_save_LIBS"
+       fi
+     fi
+
+     XML_CPPFLAGS=""
+     XML_LIBS=""
+     ifelse([$3], , :, [$3])
+  fi
+  AC_SUBST(XML_CPPFLAGS)
+  AC_SUBST(XML_LIBS)
+  rm -f conf.xmltest
+])

makebashcompletion

@@ -0,0 +1,7 @@
+#!/bin/sh -e
+
+BCPATH=doc/bash_completion
+
+for prog in nghttp nghttpd nghttpx h2load; do
+    $BCPATH/make_bash_completion.py src/$prog > $BCPATH/$prog
+done

python/nghttp2.pyx

@@ -400,12 +400,6 @@ cdef int server_on_frame_recv(cnghttp2.nghttp2_session *session,
             handler = _get_stream_user_data(session, frame.hd.stream_id)
             if not handler:
                 return 0
-            # Check required header fields. We expect that :authority
-            # or host header field.
-            if handler.scheme is None or handler.method is None or\
-               handler.host is None or handler.path is None:
-                return http2._rst_stream(frame.hd.stream_id,
-                                         cnghttp2.NGHTTP2_PROTOCOL_ERROR)
             if handler.cookies:
                 handler.headers.append((b'cookie',
                                         b'; '.join(handler.cookies)))
@@ -556,10 +550,7 @@ cdef int client_on_frame_recv(cnghttp2.nghttp2_session *session,
 
             if not handler:
                 return 0
-            # Check required header fields. We expect a status.
-            if handler.status is None:
-                return http2._rst_stream(frame.hd.stream_id,
-                                         cnghttp2.NGHTTP2_PROTOCOL_ERROR)
+            # TODO handle 1xx non-final response
             if handler.cookies:
                 handler.headers.append((b'cookie',
                                         b'; '.join(handler.cookies)))
@@ -587,10 +578,6 @@ cdef int client_on_frame_recv(cnghttp2.nghttp2_session *session,
         cnghttp2.nghttp2_session_set_stream_user_data(session, frame.push_promise.promised_stream_id,
                                                       <void*>NULL)
 
-        if push_handler.scheme is None or push_handler.method is None or\
-           push_handler.host is None or push_handler.path is None:
-            return http2._rst_stream(frame.push_promise.promised_stream_id,
-                                     cnghttp2.NGHTTP2_PROTOCOL_ERROR)
         try:
             handler.on_push_promise(push_handler)
         except:

src/HttpServer.cc

@@ -31,6 +31,7 @@
 #include <fcntl.h>
 #include <netinet/in.h>
 #include <netinet/tcp.h>
+#include <arpa/inet.h>
 
 #include <cassert>
 #include <set>
@@ -47,6 +48,7 @@
 #include "http2.h"
 #include "util.h"
 #include "ssl.h"
+#include "template.h"
 
 #ifndef O_BINARY
 #define O_BINARY (0)
@@ -76,7 +78,7 @@ void print_session_id(int64_t id) { std::cout << "[id=" << id << "] "; }
 } // namespace
 
 namespace {
-void append_nv(Stream *stream, const std::vector<nghttp2_nv> &nva) {
+template <typename Array> void append_nv(Stream *stream, const Array &nva) {
   for (size_t i = 0; i < nva.size(); ++i) {
     auto &nv = nva[i];
     auto token = http2::lookup_token(nv.name, nv.namelen);
@@ -84,7 +86,7 @@ void append_nv(Stream *stream, const std::vector<nghttp2_nv> &nva) {
       http2::index_header(stream->hdidx, token, i);
     }
     http2::add_header(stream->headers, nv.name, nv.namelen, nv.value,
-                      nv.valuelen, nv.flags & NGHTTP2_NV_FLAG_NO_INDEX);
+                      nv.valuelen, nv.flags & NGHTTP2_NV_FLAG_NO_INDEX, token);
   }
 }
 } // namespace
@@ -223,7 +225,7 @@ public:
       }
     }
     auto handler =
-        util::make_unique<Http2Handler>(this, fd, ssl, get_next_session_id());
+        make_unique<Http2Handler>(this, fd, ssl, get_next_session_id());
     handler->setup_bev();
     if (!ssl) {
       if (handler->on_connect() != 0) {
@@ -246,8 +248,7 @@ private:
 };
 
 Stream::Stream(Http2Handler *handler, int32_t stream_id)
-    : handler(handler), body_left(0), upload_left(-1), stream_id(stream_id),
-      file(-1) {
+    : handler(handler), body_left(0), stream_id(stream_id), file(-1) {
   auto config = handler->get_config();
   ev_timer_init(&rtimer, stream_timeout_cb, 0., config->stream_read_timeout);
   ev_timer_init(&wtimer, stream_timeout_cb, 0., config->stream_write_timeout);
@@ -401,11 +402,11 @@ int Http2Handler::fill_wb() {
 
 int Http2Handler::read_clear() {
   int rv;
-  uint8_t buf[8192];
+  std::array<uint8_t, 8192> buf;
 
   for (;;) {
     ssize_t nread;
-    while ((nread = read(fd_, buf, sizeof(buf))) == -1 && errno == EINTR)
+    while ((nread = read(fd_, buf.data(), buf.size())) == -1 && errno == EINTR)
       ;
     if (nread == -1) {
       if (errno == EAGAIN || errno == EWOULDBLOCK) {
@@ -416,10 +417,12 @@ int Http2Handler::read_clear() {
     if (nread == 0) {
       return -1;
     }
-    rv = nghttp2_session_mem_recv(session_, buf, nread);
+    rv = nghttp2_session_mem_recv(session_, buf.data(), nread);
     if (rv < 0) {
-      std::cerr << "nghttp2_session_mem_recv() returned error: "
-                << nghttp2_strerror(rv) << std::endl;
+      if (rv != NGHTTP2_ERR_BAD_PREFACE) {
+        std::cerr << "nghttp2_session_mem_recv() returned error: "
+                  << nghttp2_strerror(rv) << std::endl;
+      }
       return -1;
     }
   }
@@ -431,11 +434,9 @@ int Http2Handler::write_clear() {
   auto loop = sessions_->get_loop();
   for (;;) {
     if (wb_.rleft() > 0) {
-      struct iovec iov[2];
-      auto iovcnt = wb_.riovec(iov);
-
       ssize_t nwrite;
-      while ((nwrite = writev(fd_, iov, iovcnt)) == -1 && errno == EINTR)
+      while ((nwrite = write(fd_, wb_.pos, wb_.rleft())) == -1 &&
+             errno == EINTR)
         ;
       if (nwrite == -1) {
         if (errno == EAGAIN || errno == EWOULDBLOCK) {
@@ -513,12 +514,12 @@ int Http2Handler::tls_handshake() {
 }
 
 int Http2Handler::read_tls() {
-  uint8_t buf[8192];
+  std::array<uint8_t, 8192> buf;
 
   ERR_clear_error();
 
   for (;;) {
-    auto rv = SSL_read(ssl_, buf, sizeof(buf));
+    auto rv = SSL_read(ssl_, buf.data(), buf.size());
 
     if (rv == 0) {
       return -1;
@@ -538,10 +539,12 @@ int Http2Handler::read_tls() {
     }
 
     auto nread = rv;
-    rv = nghttp2_session_mem_recv(session_, buf, nread);
+    rv = nghttp2_session_mem_recv(session_, buf.data(), nread);
     if (rv < 0) {
-      std::cerr << "nghttp2_session_mem_recv() returned error: "
-                << nghttp2_strerror(rv) << std::endl;
+      if (rv != NGHTTP2_ERR_BAD_PREFACE) {
+        std::cerr << "nghttp2_session_mem_recv() returned error: "
+                  << nghttp2_strerror(rv) << std::endl;
+      }
       return -1;
     }
   }
@@ -557,11 +560,7 @@ int Http2Handler::write_tls() {
 
   for (;;) {
     if (wb_.rleft() > 0) {
-      const void *p;
-      size_t len;
-      std::tie(p, len) = wb_.get();
-
-      auto rv = SSL_write(ssl_, p, len);
+      auto rv = SSL_write(ssl_, wb_.pos, wb_.rleft());
 
       if (rv == 0) {
         return -1;
@@ -619,7 +618,7 @@ int Http2Handler::on_connect() {
   if (r != 0) {
     return r;
   }
-  nghttp2_settings_entry entry[4];
+  std::array<nghttp2_settings_entry, 4> entry;
   size_t niv = 1;
 
   entry[0].settings_id = NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS;
@@ -630,7 +629,7 @@ int Http2Handler::on_connect() {
     entry[niv].value = sessions_->get_config()->header_table_size;
     ++niv;
   }
-  r = nghttp2_submit_settings(session_, NGHTTP2_FLAG_NONE, entry, niv);
+  r = nghttp2_submit_settings(session_, NGHTTP2_FLAG_NONE, entry.data(), niv);
   if (r != 0) {
     return r;
   }
@@ -677,30 +676,29 @@ int Http2Handler::submit_file_response(const std::string &status,
                                        nghttp2_data_provider *data_prd) {
   std::string content_length = util::utos(file_length);
   std::string last_modified_str;
-  nghttp2_nv nva[] = {
-      http2::make_nv_ls(":status", status),
-      http2::make_nv_ls("server", NGHTTPD_SERVER),
-      http2::make_nv_ls("content-length", content_length),
-      http2::make_nv_ll("cache-control", "max-age=3600"),
-      http2::make_nv_ls("date", sessions_->get_cached_date()),
-      http2::make_nv_ll("", ""),
-  };
+  auto nva = make_array(http2::make_nv_ls(":status", status),
+                        http2::make_nv_ls("server", NGHTTPD_SERVER),
+                        http2::make_nv_ls("content-length", content_length),
+                        http2::make_nv_ll("cache-control", "max-age=3600"),
+                        http2::make_nv_ls("date", sessions_->get_cached_date()),
+                        http2::make_nv_ll("", ""));
   size_t nvlen = 5;
   if (last_modified != 0) {
     last_modified_str = util::http_date(last_modified);
     nva[nvlen++] = http2::make_nv_ls("last-modified", last_modified_str);
   }
-  return nghttp2_submit_response(session_, stream->stream_id, nva, nvlen,
+  return nghttp2_submit_response(session_, stream->stream_id, nva.data(), nvlen,
                                  data_prd);
 }
 
 int Http2Handler::submit_response(const std::string &status, int32_t stream_id,
                                   const Headers &headers,
                                   nghttp2_data_provider *data_prd) {
-  auto nva = std::vector<nghttp2_nv>{
-      http2::make_nv_ls(":status", status),
-      http2::make_nv_ls("server", NGHTTPD_SERVER),
-      http2::make_nv_ls("date", sessions_->get_cached_date())};
+  auto nva = std::vector<nghttp2_nv>();
+  nva.reserve(3 + headers.size());
+  nva.push_back(http2::make_nv_ls(":status", status));
+  nva.push_back(http2::make_nv_ls("server", NGHTTPD_SERVER));
+  nva.push_back(http2::make_nv_ls("date", sessions_->get_cached_date()));
   for (auto &nv : headers) {
     nva.push_back(http2::make_nv(nv.name, nv.value, nv.no_index));
   }
@@ -711,16 +709,15 @@ int Http2Handler::submit_response(const std::string &status, int32_t stream_id,
 
 int Http2Handler::submit_response(const std::string &status, int32_t stream_id,
                                   nghttp2_data_provider *data_prd) {
-  auto nva =
-      std::vector<nghttp2_nv>{http2::make_nv_ls(":status", status),
-                              http2::make_nv_ls("server", NGHTTPD_SERVER)};
+  auto nva = make_array(http2::make_nv_ls(":status", status),
+                        http2::make_nv_ls("server", NGHTTPD_SERVER));
   return nghttp2_submit_response(session_, stream_id, nva.data(), nva.size(),
                                  data_prd);
 }
 
 int Http2Handler::submit_non_final_response(const std::string &status,
                                             int32_t stream_id) {
-  auto nva = std::vector<nghttp2_nv>{http2::make_nv_ls(":status", status)};
+  auto nva = make_array(http2::make_nv_ls(":status", status));
   return nghttp2_submit_headers(session_, NGHTTP2_FLAG_NONE, stream_id, nullptr,
                                 nva.data(), nva.size(), nullptr);
 }
@@ -735,12 +732,12 @@ int Http2Handler::submit_push_promise(Stream *stream,
         http2::get_header(stream->hdidx, http2::HD_HOST, stream->headers);
   }
 
-  auto nva = std::vector<nghttp2_nv>{
-      http2::make_nv_ll(":method", "GET"),
-      http2::make_nv_ls(":path", push_path),
-      get_config()->no_tls ? http2::make_nv_ll(":scheme", "http")
-                           : http2::make_nv_ll(":scheme", "https"),
-      http2::make_nv_ls(":authority", authority->value)};
+  auto nva =
+      make_array(http2::make_nv_ll(":method", "GET"),
+                 http2::make_nv_ls(":path", push_path),
+                 get_config()->no_tls ? http2::make_nv_ll(":scheme", "http")
+                                      : http2::make_nv_ll(":scheme", "https"),
+                 http2::make_nv_ls(":authority", authority->value));
 
   auto promised_stream_id = nghttp2_submit_push_promise(
       session_, NGHTTP2_FLAG_END_HEADERS, stream->stream_id, nva.data(),
@@ -750,7 +747,7 @@ int Http2Handler::submit_push_promise(Stream *stream,
     return promised_stream_id;
   }
 
-  auto promised_stream = util::make_unique<Stream>(this, promised_stream_id);
+  auto promised_stream = make_unique<Stream>(this, promised_stream_id);
 
   append_nv(promised_stream.get(), nva);
   add_stream(promised_stream_id, std::move(promised_stream));
@@ -1024,38 +1021,11 @@ int on_header_callback(nghttp2_session *session, const nghttp2_frame *frame,
     return 0;
   }
 
-  if (!http2::check_nv(name, namelen, value, valuelen)) {
-    hd->submit_rst_stream(stream, NGHTTP2_PROTOCOL_ERROR);
-    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-  }
-
   auto token = http2::lookup_token(name, namelen);
 
-  if (name[0] == ':') {
-    if ((!stream->headers.empty() &&
-         stream->headers.back().name.c_str()[0] != ':') ||
-        !http2::check_http2_request_pseudo_header(stream->hdidx, token)) {
-
-      hd->submit_rst_stream(stream, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
-  }
-
-  if (!http2::http2_header_allowed(token)) {
-    hd->submit_rst_stream(stream, NGHTTP2_PROTOCOL_ERROR);
-    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-  }
-
-  if (token == http2::HD_CONTENT_LENGTH) {
-    auto upload_left = util::parse_uint(value, valuelen);
-    if (upload_left != -1) {
-      stream->upload_left = upload_left;
-    }
-  }
-
   http2::index_header(stream->hdidx, token, stream->headers.size());
   http2::add_header(stream->headers, name, namelen, value, valuelen,
-                    flags & NGHTTP2_NV_FLAG_NO_INDEX);
+                    flags & NGHTTP2_NV_FLAG_NO_INDEX, token);
   return 0;
 }
 } // namespace
@@ -1070,7 +1040,7 @@ int on_begin_headers_callback(nghttp2_session *session,
     return 0;
   }
 
-  auto stream = util::make_unique<Stream>(hd, frame->hd.stream_id);
+  auto stream = make_unique<Stream>(hd, frame->hd.stream_id);
 
   add_stream_read_timeout(stream.get());
 
@@ -1098,10 +1068,6 @@ int hd_on_frame_recv_callback(nghttp2_session *session,
 
     if (frame->hd.flags & NGHTTP2_FLAG_END_STREAM) {
       remove_stream_read_timeout(stream);
-      if (stream->upload_left > 0) {
-        hd->submit_rst_stream(stream, NGHTTP2_PROTOCOL_ERROR);
-        return 0;
-      }
       if (!hd->get_config()->early_response) {
         prepare_response(stream, hd);
       }
@@ -1119,15 +1085,10 @@ int hd_on_frame_recv_callback(nghttp2_session *session,
 
     if (frame->headers.cat == NGHTTP2_HCAT_REQUEST) {
 
-      if (!http2::http2_mandatory_request_headers_presence(stream->hdidx)) {
-        hd->submit_rst_stream(stream, NGHTTP2_PROTOCOL_ERROR);
-        return 0;
-      }
-
       auto expect100 =
           http2::get_header(stream->hdidx, http2::HD_EXPECT, stream->headers);
 
-      if (expect100 && util::strieq("100-continue", expect100->value.c_str())) {
+      if (expect100 && util::strieq_l("100-continue", expect100->value)) {
         hd->submit_non_final_response("100", frame->hd.stream_id);
       }
 
@@ -1138,10 +1099,6 @@ int hd_on_frame_recv_callback(nghttp2_session *session,
 
     if (frame->hd.flags & NGHTTP2_FLAG_END_STREAM) {
       remove_stream_read_timeout(stream);
-      if (stream->upload_left > 0) {
-        hd->submit_rst_stream(stream, NGHTTP2_PROTOCOL_ERROR);
-        return 0;
-      }
       if (!hd->get_config()->early_response) {
         prepare_response(stream, hd);
       }
@@ -1238,15 +1195,6 @@ int on_data_chunk_recv_callback(nghttp2_session *session, uint8_t flags,
 
   // TODO Handle POST
 
-  if (stream->upload_left != -1) {
-    if (stream->upload_left < static_cast<int64_t>(len)) {
-      stream->upload_left = -1;
-      hd->submit_rst_stream(stream, NGHTTP2_PROTOCOL_ERROR);
-      return 0;
-    }
-    stream->upload_left -= len;
-  }
-
   add_stream_read_timeout(stream);
 
   return 0;
@@ -1349,10 +1297,10 @@ public:
       if (config_->verbose) {
         std::cerr << "spawning thread #" << i << std::endl;
       }
-      auto worker = util::make_unique<Worker>();
+      auto worker = make_unique<Worker>();
       auto loop = ev_loop_new(0);
       worker->sessions =
-          util::make_unique<Sessions>(loop, config_, sessions_->get_ssl_ctx());
+          make_unique<Sessions>(loop, config_, sessions_->get_ssl_ctx());
       ev_async_init(&worker->w, worker_acceptcb);
       worker->w.data = worker.get();
       ev_async_start(loop, &worker->w);
@@ -1465,6 +1413,7 @@ int start_listen(struct ev_loop *loop, Sessions *sessions,
   addrinfo hints;
   int r;
   bool ok = false;
+  const char *addr = nullptr;
 
   auto acceptor = std::make_shared<AcceptHandler>(sessions, config);
   auto service = util::utos(config->port);
@@ -1477,12 +1426,17 @@ int start_listen(struct ev_loop *loop, Sessions *sessions,
   hints.ai_flags |= AI_ADDRCONFIG;
 #endif // AI_ADDRCONFIG
 
+  if (!config->address.empty()) {
+    addr = config->address.c_str();
+  }
+
   addrinfo *res, *rp;
-  r = getaddrinfo(nullptr, service.c_str(), &hints, &res);
+  r = getaddrinfo(addr, service.c_str(), &hints, &res);
   if (r != 0) {
     std::cerr << "getaddrinfo() failed: " << gai_strerror(r) << std::endl;
     return -1;
   }
+
   for (rp = res; rp; rp = rp->ai_next) {
     int fd = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
     if (fd == -1) {
@@ -1508,8 +1462,9 @@ int start_listen(struct ev_loop *loop, Sessions *sessions,
       new ListenEventHandler(sessions, fd, acceptor);
 
       if (config->verbose) {
-        std::cout << (rp->ai_family == AF_INET ? "IPv4" : "IPv6")
-                  << ": listen on port " << config->port << std::endl;
+        std::string s = util::numeric_name(rp->ai_addr, rp->ai_addrlen);
+        std::cout << (rp->ai_family == AF_INET ? "IPv4" : "IPv6") << ": listen "
+                  << s << ":" << config->port << std::endl;
       }
       ok = true;
       continue;

src/HttpServer.h

@@ -44,7 +44,7 @@
 #include <nghttp2/nghttp2.h>
 
 #include "http2.h"
-#include "ringbuf.h"
+#include "buffer.h"
 
 namespace nghttp2 {
 
@@ -55,6 +55,7 @@ struct Config {
   std::string private_key_file;
   std::string cert_file;
   std::string dh_param_file;
+  std::string address;
   ev_tstamp stream_read_timeout;
   ev_tstamp stream_write_timeout;
   nghttp2_option *session_option;
@@ -81,10 +82,9 @@ struct Stream {
   ev_timer rtimer;
   ev_timer wtimer;
   int64_t body_left;
-  int64_t upload_left;
   int32_t stream_id;
   int file;
-  int hdidx[http2::HD_MAXIDX];
+  http2::HeaderIndex hdidx;
   Stream(Http2Handler *handler, int32_t stream_id);
   ~Stream();
 };
@@ -143,7 +143,7 @@ private:
   ev_io rev_;
   ev_timer settings_timerev_;
   std::map<int32_t, std::unique_ptr<Stream>> id2stream_;
-  RingBuf<65536> wb_;
+  Buffer<65536> wb_;
   std::function<int(Http2Handler &)> read_, write_;
   int64_t session_id_;
   nghttp2_session *session_;

src/Makefile.am

@@ -76,8 +76,7 @@ nghttp_SOURCES = ${HELPER_OBJECTS} ${HELPER_HFILES} nghttp.cc nghttp.h \
 
 nghttpd_SOURCES = ${HELPER_OBJECTS} ${HELPER_HFILES} nghttpd.cc \
 	ssl.cc ssl.h \
-	HttpServer.cc HttpServer.h \
-	ringbuf.h
+	HttpServer.cc HttpServer.h
 
 bin_PROGRAMS += h2load
 
@@ -115,11 +114,12 @@ NGHTTPX_SRCS = \
 	shrpx_io_control.cc shrpx_io_control.h \
 	shrpx_ssl.cc shrpx_ssl.h \
 	shrpx_worker.cc shrpx_worker.h \
-	shrpx_worker_config.cc shrpx_worker_config.h \
+	shrpx_log_config.cc shrpx_log_config.h \
 	shrpx_connect_blocker.cc shrpx_connect_blocker.h \
 	shrpx_downstream_connection_pool.cc shrpx_downstream_connection_pool.h \
 	shrpx_rate_limit.cc shrpx_rate_limit.h \
-	ringbuf.h memchunk.h
+	shrpx_connection.cc shrpx_connection.h \
+	buffer.h memchunk.h template.h
 
 if HAVE_SPDYLAY
 NGHTTPX_SRCS += shrpx_spdy_upstream.cc shrpx_spdy_upstream.h
@@ -141,7 +141,7 @@ nghttpx_unittest_SOURCES = shrpx-unittest.cc \
 	util_test.cc util_test.h \
 	nghttp2_gzip_test.c nghttp2_gzip_test.h \
 	nghttp2_gzip.c nghttp2_gzip.h \
-	ringbuf_test.cc ringbuf_test.h \
+	buffer_test.cc buffer_test.h \
 	memchunk_test.cc memchunk_test.h
 nghttpx_unittest_CPPFLAGS = ${AM_CPPFLAGS}\
 	-DNGHTTP2_TESTS_DIR=\"$(top_srcdir)/tests\"

src/app_helper.cc

@@ -408,8 +408,11 @@ int verbose_on_header_callback(nghttp2_session *session,
                    namelen, valuelen};
 
   print_timer();
-  fprintf(outfile, " recv (stream_id=%d, noind=%d) ", frame->hd.stream_id,
-          (flags & NGHTTP2_NV_FLAG_NO_INDEX) != 0);
+  fprintf(outfile, " recv (stream_id=%d", frame->hd.stream_id);
+  if (flags & NGHTTP2_NV_FLAG_NO_INDEX) {
+    fprintf(outfile, ", sensitive");
+  }
+  fprintf(outfile, ") ");
 
   print_nv(&nv);
   fflush(outfile);

src/asio_connection.h

@@ -158,7 +158,7 @@ private:
   /// Buffer for incoming data.
   boost::array<uint8_t, 8192> buffer_;
 
-  boost::array<uint8_t, 16394> outbuf_;
+  boost::array<uint8_t, 65536> outbuf_;
 
   bool writing_;
 };

src/asio_http2_handler.cc

@@ -28,12 +28,13 @@
 
 #include "http2.h"
 #include "util.h"
+#include "template.h"
 
 namespace nghttp2 {
 
 namespace asio_http2 {
 
-channel::channel() : impl_(util::make_unique<channel_impl>()) {}
+channel::channel() : impl_(make_unique<channel_impl>()) {}
 
 void channel::post(void_cb cb) { impl_->post(std::move(cb)); }
 
@@ -51,7 +52,7 @@ namespace server {
 
 extern std::shared_ptr<std::string> cached_date;
 
-request::request() : impl_(util::make_unique<request_impl>()) {}
+request::request() : impl_(make_unique<request_impl>()) {}
 
 const std::vector<header> &request::headers() const { return impl_->headers(); }
 
@@ -84,7 +85,7 @@ bool request::run_task(thread_cb start) {
 
 request_impl &request::impl() { return *impl_; }
 
-response::response() : impl_(util::make_unique<response_impl>()) {}
+response::response() : impl_(make_unique<response_impl>()) {}
 
 void response::write_head(unsigned int status_code,
                           std::vector<header> headers) {
@@ -335,54 +336,25 @@ int on_header_callback(nghttp2_session *session, const nghttp2_frame *frame,
     return 0;
   }
 
-  if (!nghttp2_check_header_name(name, namelen) ||
-      !nghttp2_check_header_value(value, valuelen)) {
-    stream_error(session, stream_id, NGHTTP2_PROTOCOL_ERROR);
-
-    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-  }
-
   auto &req = stream->get_request()->impl();
 
-  if (name[0] == ':' && !req.headers().empty()) {
-    stream_error(session, stream_id, NGHTTP2_PROTOCOL_ERROR);
-    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-  }
-
-  if (util::streq(":method", name, namelen)) {
-    if (!req.method().empty()) {
-      stream_error(session, stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
+  switch (nghttp2::http2::lookup_token(name, namelen)) {
+  case nghttp2::http2::HD__METHOD:
     req.method(std::string(value, value + valuelen));
-  } else if (util::streq(":scheme", name, namelen)) {
-    if (!req.scheme().empty()) {
-      stream_error(session, stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
+    break;
+  case nghttp2::http2::HD__SCHEME:
     req.scheme(std::string(value, value + valuelen));
-  } else if (util::streq(":authority", name, namelen)) {
-    if (!req.authority().empty()) {
-      stream_error(session, stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
+    break;
+  case nghttp2::http2::HD__AUTHORITY:
     req.authority(std::string(value, value + valuelen));
-  } else if (util::streq(":path", name, namelen)) {
-    if (!req.path().empty()) {
-      stream_error(session, stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
+    break;
+  case nghttp2::http2::HD__PATH:
     req.path(std::string(value, value + valuelen));
-  } else {
-    if (name[0] == ':') {
-      stream_error(session, stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-    }
-
-    if (util::streq("host", name, namelen)) {
-      req.host(std::string(value, value + valuelen));
-    }
-
+    break;
+  case nghttp2::http2::HD_HOST:
+    req.host(std::string(value, value + valuelen));
+  // fall through
+  default:
     req.add_header(std::string(name, name + namelen),
                    std::string(value, value + valuelen));
   }
@@ -415,12 +387,6 @@ int on_frame_recv_callback(nghttp2_session *session, const nghttp2_frame *frame,
 
     auto &req = stream->get_request()->impl();
 
-    if (req.method().empty() || req.scheme().empty() || req.path().empty() ||
-        (req.authority().empty() && req.host().empty())) {
-      stream_error(session, frame->hd.stream_id, NGHTTP2_PROTOCOL_ERROR);
-      return 0;
-    }
-
     if (req.host().empty()) {
       req.host(req.authority());
     }
@@ -524,7 +490,7 @@ int http2_handler::start() {
     return -1;
   }
 
-  auto cb_del = util::defer(callbacks, nghttp2_session_callbacks_del);
+  auto cb_del = defer(nghttp2_session_callbacks_del, callbacks);
 
   nghttp2_session_callbacks_set_on_begin_headers_callback(
       callbacks, on_begin_headers_callback);
@@ -547,7 +513,7 @@ int http2_handler::start() {
     return -1;
   }
 
-  auto opt_del = util::defer(option, nghttp2_option_del);
+  auto opt_del = defer(nghttp2_option_del, option);
 
   nghttp2_option_set_recv_client_preface(option, 1);
 

src/asio_http2_handler.h

@@ -211,7 +211,7 @@ public:
     len = 0;
 
     if (buf_) {
-      std::copy(buf_, buf_ + buflen_, std::begin(buffer));
+      std::copy_n(buf_, buflen_, std::begin(buffer));
 
       len += buflen_;
 
@@ -237,7 +237,7 @@ public:
         break;
       }
 
-      std::copy(data, data + nread, std::begin(buffer) + len);
+      std::copy_n(data, nread, std::begin(buffer) + len);
 
       len += nread;
     }

src/asio_http2_impl.cc

@@ -33,6 +33,7 @@
 #include "asio_server.h"
 #include "util.h"
 #include "ssl.h"
+#include "template.h"
 
 namespace nghttp2 {
 
@@ -40,7 +41,7 @@ namespace asio_http2 {
 
 namespace server {
 
-http2::http2() : impl_(util::make_unique<http2_impl>()) {}
+http2::http2() : impl_(make_unique<http2_impl>()) {}
 
 http2::~http2() {}
 
@@ -75,7 +76,7 @@ void http2_impl::listen(const std::string &address, uint16_t port,
   std::unique_ptr<boost::asio::ssl::context> ssl_ctx;
 
   if (!private_key_file_.empty() && !certificate_file_.empty()) {
-    ssl_ctx = util::make_unique<boost::asio::ssl::context>(
+    ssl_ctx = make_unique<boost::asio::ssl::context>(
         boost::asio::ssl::context::sslv23);
 
     ssl_ctx->use_private_key_file(private_key_file_,
@@ -134,10 +135,10 @@ void http2_impl::backlog(int backlog) { backlog_ = backlog; }
 
 } // namespace server
 
-template <typename T, typename F>
-std::shared_ptr<util::Defer<T, F>> defer_shared(T &&t, F f) {
-  return std::make_shared<util::Defer<T, F>>(std::forward<T>(t),
-                                             std::forward<F>(f));
+template <typename F, typename... T>
+std::shared_ptr<Defer<F, T...>> defer_shared(F &&f, T &&... t) {
+  return std::make_shared<Defer<F, T...>>(std::forward<F>(f),
+                                          std::forward<T>(t)...);
 }
 
 read_cb file_reader(const std::string &path) {
@@ -150,7 +151,7 @@ read_cb file_reader(const std::string &path) {
 }
 
 read_cb file_reader_from_fd(int fd) {
-  auto d = defer_shared(static_cast<int>(fd), close);
+  auto d = defer_shared(close, fd);
 
   return [fd, d](uint8_t *buf, size_t len) -> read_cb::result_type {
     int rv;

src/buffer.h

@@ -0,0 +1,68 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2014 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef BUFFER_H
+#define BUFFER_H
+
+#include "nghttp2_config.h"
+
+#include <cstring>
+#include <algorithm>
+#include <array>
+
+namespace nghttp2 {
+
+template <size_t N> struct Buffer {
+  Buffer() : pos(std::begin(buf)), last(pos) {}
+  // Returns the number of bytes to read.
+  size_t rleft() const { return last - pos; }
+  // Returns the number of bytes this buffer can store.
+  size_t wleft() const { return std::end(buf) - last; }
+  // Writes up to min(wleft(), |count|) bytes from buffer pointed by
+  // |src|.  Returns number of bytes written.
+  size_t write(const void *src, size_t count) {
+    count = std::min(count, wleft());
+    auto p = static_cast<const uint8_t *>(src);
+    last = std::copy_n(p, count, last);
+    return count;
+  }
+  size_t write(size_t count) {
+    count = std::min(count, wleft());
+    last += count;
+    return count;
+  }
+  // Drains min(rleft(), |count|) bytes from start of the buffer.
+  size_t drain(size_t count) {
+    count = std::min(count, rleft());
+    pos += count;
+    return count;
+  }
+  void reset() { pos = last = std::begin(buf); }
+  std::array<uint8_t, N> buf;
+  uint8_t *pos, *last;
+};
+
+} // namespace nghttp2
+
+#endif // BUFFER_H

src/buffer_test.cc

@@ -0,0 +1,78 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2015 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "buffer_test.h"
+
+#include <cstring>
+#include <iostream>
+#include <tuple>
+
+#include <CUnit/CUnit.h>
+
+#include <nghttp2/nghttp2.h>
+
+#include "buffer.h"
+
+namespace nghttp2 {
+
+void test_buffer_write(void) {
+  Buffer<16> b;
+  CU_ASSERT(0 == b.rleft());
+  CU_ASSERT(16 == b.wleft());
+
+  b.write("012", 3);
+
+  CU_ASSERT(3 == b.rleft());
+  CU_ASSERT(13 == b.wleft());
+  CU_ASSERT(b.pos == std::begin(b.buf));
+
+  b.drain(3);
+
+  CU_ASSERT(0 == b.rleft());
+  CU_ASSERT(13 == b.wleft());
+  CU_ASSERT(3 == b.pos - std::begin(b.buf));
+
+  auto n = b.write("0123456789ABCDEF", 16);
+
+  CU_ASSERT(n == 13);
+
+  CU_ASSERT(13 == b.rleft());
+  CU_ASSERT(0 == b.wleft());
+  CU_ASSERT(3 == b.pos - std::begin(b.buf));
+  CU_ASSERT(0 == memcmp(b.pos, "0123456789ABC", 13));
+
+  b.reset();
+
+  CU_ASSERT(0 == b.rleft());
+  CU_ASSERT(16 == b.wleft());
+  CU_ASSERT(b.pos == std::begin(b.buf));
+
+  b.write(5);
+
+  CU_ASSERT(5 == b.rleft());
+  CU_ASSERT(11 == b.wleft());
+  CU_ASSERT(b.pos == std::begin(b.buf));
+}
+
+} // namespace nghttp2

src/buffer_test.h

@@ -1,7 +1,7 @@
 /*
  * nghttp2 - HTTP/2 C Library
  *
- * Copyright (c) 2014 Tatsuhiro Tsujikawa
+ * Copyright (c) 2015 Tatsuhiro Tsujikawa
  *
  * Permission is hereby granted, free of charge, to any person obtaining
  * a copy of this software and associated documentation files (the
@@ -22,14 +22,13 @@
  * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
  * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
  */
-#ifndef RINGBUF_TEST_H
-#define RINGBUF_TEST_H
+#ifndef BUFFER_TEST_H
+#define BUFFER_TEST_H
 
 namespace nghttp2 {
 
-void test_ringbuf_write(void);
-void test_ringbuf_iovec(void);
+void test_buffer_write(void);
 
 } // namespace nghttp2
 
-#endif // RINGBUF_TEST_H
+#endif // BUFFER_TEST_H

src/h2load.cc

@@ -33,6 +33,7 @@
 #include <cassert>
 #include <cstdlib>
 #include <iostream>
+#include <iomanip>
 #include <fstream>
 #include <chrono>
 #include <thread>
@@ -54,6 +55,7 @@
 #include "ssl.h"
 #include "http2.h"
 #include "util.h"
+#include "template.h"
 
 using namespace nghttp2;
 
@@ -93,16 +95,14 @@ void debug_nextproto_error() {
 }
 } // namespace
 
-Stream::Stream() : status_success(-1) {}
+RequestStat::RequestStat() : completed(false) {}
 
-namespace {
-void readcb(struct ev_loop *loop, ev_io *w, int revents) {
-  auto client = static_cast<Client *>(w->data);
-  if (client->do_read() != 0) {
-    client->fail();
-  }
-}
-} // namespace
+Stats::Stats(size_t req_todo)
+    : req_todo(0), req_started(0), req_done(0), req_success(0),
+      req_status_success(0), req_failed(0), req_error(0), bytes_total(0),
+      bytes_head(0), bytes_body(0), status(), req_stats(req_todo) {}
+
+Stream::Stream() : status_success(-1) {}
 
 namespace {
 void writecb(struct ev_loop *loop, ev_io *w, int revents) {
@@ -123,6 +123,20 @@ void writecb(struct ev_loop *loop, ev_io *w, int revents) {
 }
 } // namespace
 
+namespace {
+void readcb(struct ev_loop *loop, ev_io *w, int revents) {
+  auto client = static_cast<Client *>(w->data);
+  if (client->do_read() != 0) {
+    client->fail();
+    return;
+  }
+  if (ev_is_active(&client->wev)) {
+    writecb(loop, &client->wev, revents);
+    // client->disconnect() and client->fail() may be called
+  }
+}
+} // namespace
+
 Client::Client(Worker *worker, size_t req_todo)
     : worker(worker), ssl(nullptr), next_addr(config.addrs), reqidx(0),
       state(CLIENT_IDLE), req_todo(req_todo), req_started(0), req_done(0),
@@ -217,8 +231,8 @@ void Client::disconnect() {
 }
 
 void Client::submit_request() {
-  session->submit_request();
-  ++worker->stats.req_started;
+  auto req_stat = &worker->stats.req_stats[worker->stats.req_started++];
+  session->submit_request(req_stat);
   ++req_started;
 }
 
@@ -271,7 +285,7 @@ void print_server_tmp_key(SSL *ssl) {
     return;
   }
 
-  auto key_del = util::defer(key, EVP_PKEY_free);
+  auto key_del = defer(EVP_PKEY_free, key);
 
   std::cout << "Server Temp Key: ";
 
@@ -284,7 +298,7 @@ void print_server_tmp_key(SSL *ssl) {
     break;
   case EVP_PKEY_EC: {
     auto ec = EVP_PKEY_get1_EC_KEY(key);
-    auto ec_del = util::defer(ec, EC_KEY_free);
+    auto ec_del = defer(EC_KEY_free, ec);
     auto nid = EC_GROUP_get_curve_name(EC_KEY_get0_group(ec));
     auto cname = EC_curve_nid2nist(nid);
     if (!cname) {
@@ -322,7 +336,7 @@ void Client::on_header(int32_t stream_id, const uint8_t *name, size_t namelen,
   }
   auto &stream = (*itr).second;
   if (stream.status_success == -1 && namelen == 7 &&
-      util::streq(":status", 7, name, namelen)) {
+      util::streq_l(":status", name, namelen)) {
     int status = 0;
     for (size_t i = 0; i < valuelen; ++i) {
       if ('0' <= value[i] && value[i] <= '9') {
@@ -352,11 +366,17 @@ void Client::on_header(int32_t stream_id, const uint8_t *name, size_t namelen,
   }
 }
 
-void Client::on_stream_close(int32_t stream_id, bool success) {
+void Client::on_stream_close(int32_t stream_id, bool success,
+                             RequestStat *req_stat) {
+  req_stat->stream_close_time = std::chrono::steady_clock::now();
+  if (success) {
+    req_stat->completed = true;
+    ++worker->stats.req_success;
+  }
   ++worker->stats.req_done;
   ++req_done;
   if (success && streams[stream_id].status_success == 1) {
-    ++worker->stats.req_success;
+    ++worker->stats.req_status_success;
   } else {
     ++worker->stats.req_failed;
   }
@@ -385,13 +405,13 @@ int Client::on_connect() {
     for (int i = 0; i < 2; ++i) {
       if (next_proto) {
         if (util::check_h2_is_selected(next_proto, next_proto_len)) {
-          session = util::make_unique<Http2Session>(this);
+          session = make_unique<Http2Session>(this);
         } else {
 #ifdef HAVE_SPDYLAY
           auto spdy_version =
               spdylay_npn_get_version(next_proto, next_proto_len);
           if (spdy_version) {
-            session = util::make_unique<SpdySession>(this, spdy_version);
+            session = make_unique<SpdySession>(this, spdy_version);
           } else {
             debug_nextproto_error();
             fail();
@@ -420,17 +440,17 @@ int Client::on_connect() {
   } else {
     switch (config.no_tls_proto) {
     case Config::PROTO_HTTP2:
-      session = util::make_unique<Http2Session>(this);
+      session = make_unique<Http2Session>(this);
       break;
 #ifdef HAVE_SPDYLAY
     case Config::PROTO_SPDY2:
-      session = util::make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY2);
+      session = make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY2);
       break;
     case Config::PROTO_SPDY3:
-      session = util::make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY3);
+      session = make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY3);
       break;
     case Config::PROTO_SPDY3_1:
-      session = util::make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY3_1);
+      session = make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY3_1);
       break;
 #endif // HAVE_SPDYLAY
     default:
@@ -501,11 +521,8 @@ int Client::read_clear() {
 int Client::write_clear() {
   for (;;) {
     if (wb.rleft() > 0) {
-      struct iovec iov[2];
-      auto iovcnt = wb.riovec(iov);
-
       ssize_t nwrite;
-      while ((nwrite = writev(fd, iov, iovcnt)) == -1 && errno == EINTR)
+      while ((nwrite = write(fd, wb.pos, wb.rleft())) == -1 && errno == EINTR)
         ;
       if (nwrite == -1) {
         if (errno == EAGAIN || errno == EWOULDBLOCK) {
@@ -517,12 +534,11 @@ int Client::write_clear() {
       wb.drain(nwrite);
       continue;
     }
-
+    wb.reset();
     if (on_write() != 0) {
       return -1;
     }
     if (wb.rleft() == 0) {
-      wb.reset();
       break;
     }
   }
@@ -627,11 +643,7 @@ int Client::write_tls() {
 
   for (;;) {
     if (wb.rleft() > 0) {
-      const void *p;
-      size_t len;
-      std::tie(p, len) = wb.get();
-
-      auto rv = SSL_write(ssl, p, len);
+      auto rv = SSL_write(ssl, wb.pos, wb.rleft());
 
       if (rv == 0) {
         return -1;
@@ -655,6 +667,7 @@ int Client::write_tls() {
 
       continue;
     }
+    wb.reset();
     if (on_write() != 0) {
       return -1;
     }
@@ -668,12 +681,16 @@ int Client::write_tls() {
   return 0;
 }
 
+void Client::record_request_time(RequestStat *req_stat) {
+  req_stat->request_time = std::chrono::steady_clock::now();
+}
+
 void Client::signal_write() { ev_io_start(worker->loop, &wev); }
 
 Worker::Worker(uint32_t id, SSL_CTX *ssl_ctx, size_t req_todo, size_t nclients,
                Config *config)
-    : stats{0}, loop(ev_loop_new(0)), ssl_ctx(ssl_ctx), config(config), id(id),
-      tls_info_report_done(false) {
+    : stats(req_todo), loop(ev_loop_new(0)), ssl_ctx(ssl_ctx), config(config),
+      id(id), tls_info_report_done(false) {
   stats.req_todo = req_todo;
   progress_interval = std::max((size_t)1, req_todo / 10);
 
@@ -686,7 +703,7 @@ Worker::Worker(uint32_t id, SSL_CTX *ssl_ctx, size_t req_todo, size_t nclients,
       ++req_todo;
       --nreqs_rem;
     }
-    clients.push_back(util::make_unique<Client>(this, req_todo));
+    clients.push_back(make_unique<Client>(this, req_todo));
   }
 }
 
@@ -707,6 +724,74 @@ void Worker::run() {
   ev_run(loop, 0);
 }
 
+namespace {
+double within_sd(const std::vector<std::unique_ptr<Worker>> &workers,
+                 const std::chrono::microseconds &mean,
+                 const std::chrono::microseconds &sd, size_t n) {
+  auto upper = mean.count() + sd.count();
+  auto lower = mean.count() - sd.count();
+  size_t m = 0;
+  for (const auto &w : workers) {
+    for (const auto &req_stat : w->stats.req_stats) {
+      if (!req_stat.completed) {
+        continue;
+      }
+      auto t = std::chrono::duration_cast<std::chrono::microseconds>(
+          req_stat.stream_close_time - req_stat.request_time);
+      if (lower <= t.count() && t.count() <= upper) {
+        ++m;
+      }
+    }
+  }
+  return (m / static_cast<double>(n)) * 100;
+}
+} // namespace
+
+namespace {
+TimeStats
+process_time_stats(const std::vector<std::unique_ptr<Worker>> &workers) {
+  auto ts = TimeStats();
+  int64_t sum = 0;
+  size_t n = 0;
+
+  ts.time_min = std::chrono::microseconds::max();
+  ts.time_max = std::chrono::microseconds::min();
+  ts.within_sd = 0.;
+
+  // standard deviation calculated using Rapid calculation method:
+  // http://en.wikipedia.org/wiki/Standard_deviation#Rapid_calculation_methods
+  double a = 0, q = 0;
+  for (const auto &w : workers) {
+    for (const auto &req_stat : w->stats.req_stats) {
+      if (!req_stat.completed) {
+        continue;
+      }
+      ++n;
+      auto t = std::chrono::duration_cast<std::chrono::microseconds>(
+          req_stat.stream_close_time - req_stat.request_time);
+      ts.time_min = std::min(ts.time_min, t);
+      ts.time_max = std::max(ts.time_max, t);
+      sum += t.count();
+
+      auto na = a + (t.count() - a) / n;
+      q = q + (t.count() - a) * (t.count() - na);
+      a = na;
+    }
+  }
+  if (n == 0) {
+    ts.time_max = ts.time_min = std::chrono::microseconds::zero();
+    return ts;
+  }
+
+  ts.time_mean = std::chrono::microseconds(sum / n);
+  ts.time_sd = std::chrono::microseconds(
+      static_cast<std::chrono::microseconds::rep>(sqrt(q / n)));
+
+  ts.within_sd = within_sd(workers, ts.time_mean, ts.time_sd, n);
+  return ts;
+}
+} // namespace
+
 namespace {
 void resolve_host() {
   int rv;
@@ -1117,8 +1202,8 @@ int main(int argc, char **argv) {
   auto proto_list = util::get_default_alpn();
 #ifdef HAVE_SPDYLAY
   static const char spdy_proto_list[] = "\x8spdy/3.1\x6spdy/3\x6spdy/2";
-  std::copy(spdy_proto_list, spdy_proto_list + sizeof(spdy_proto_list) - 1,
-            std::back_inserter(proto_list));
+  std::copy_n(spdy_proto_list, sizeof(spdy_proto_list) - 1,
+              std::back_inserter(proto_list));
 #endif // HAVE_SPDYLAY
   SSL_CTX_set_alpn_protos(ssl_ctx, proto_list.data(), proto_list.size());
 #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
@@ -1162,7 +1247,7 @@ int main(int argc, char **argv) {
 
   // list overridalbe headers
   auto override_hdrs =
-      std::vector<std::string>{":authority", ":host", ":method", ":scheme"};
+      make_array<std::string>(":authority", ":host", ":method", ":scheme");
 
   for (auto &kv : config.custom_headers) {
     if (std::find(std::begin(override_hdrs), std::end(override_hdrs),
@@ -1229,23 +1314,22 @@ int main(int argc, char **argv) {
 
   auto start = std::chrono::steady_clock::now();
 
+  std::vector<std::unique_ptr<Worker>> workers;
+  workers.reserve(config.nthreads - 1);
+
 #ifndef NOTHREADS
-  std::vector<std::future<Stats>> futures;
+  std::vector<std::future<void>> futures;
   for (size_t i = 0; i < config.nthreads - 1; ++i) {
     auto nreqs = nreqs_per_thread + (nreqs_rem-- > 0);
     auto nclients = nclients_per_thread + (nclients_rem-- > 0);
     std::cout << "spawning thread #" << i << ": " << nclients
               << " concurrent clients, " << nreqs << " total requests"
               << std::endl;
-    auto worker =
-        util::make_unique<Worker>(i, ssl_ctx, nreqs, nclients, &config);
-    futures.push_back(std::async(std::launch::async,
-                                 [](std::unique_ptr<Worker> worker) {
-                                   worker->run();
-
-                                   return worker->stats;
-                                 },
-                                 std::move(worker)));
+    workers.push_back(
+        make_unique<Worker>(i, ssl_ctx, nreqs, nclients, &config));
+    auto &worker = workers.back();
+    futures.push_back(
+        std::async(std::launch::async, [&worker]() { worker->run(); }));
   }
 #endif // NOTHREADS
 
@@ -1254,73 +1338,81 @@ int main(int argc, char **argv) {
   std::cout << "spawning thread #" << (config.nthreads - 1) << ": "
             << nclients_last << " concurrent clients, " << nreqs_last
             << " total requests" << std::endl;
-  Worker worker(config.nthreads - 1, ssl_ctx, nreqs_last, nclients_last,
-                &config);
-  worker.run();
+  workers.push_back(make_unique<Worker>(config.nthreads - 1, ssl_ctx,
+                                        nreqs_last, nclients_last, &config));
+  workers.back()->run();
 
 #ifndef NOTHREADS
   for (auto &fut : futures) {
-    auto stats = fut.get();
-
-    worker.stats.req_todo += stats.req_todo;
-    worker.stats.req_started += stats.req_started;
-    worker.stats.req_done += stats.req_done;
-    worker.stats.req_success += stats.req_success;
-    worker.stats.req_failed += stats.req_failed;
-    worker.stats.req_error += stats.req_error;
-    worker.stats.bytes_total += stats.bytes_total;
-    worker.stats.bytes_head += stats.bytes_head;
-    worker.stats.bytes_body += stats.bytes_body;
-
-    for (size_t i = 0; i < util::array_size(stats.status); ++i) {
-      worker.stats.status[i] += stats.status[i];
-    }
+    fut.get();
   }
 #endif // NOTHREADS
 
   auto end = std::chrono::steady_clock::now();
-  auto duration = std::chrono::duration_cast<std::chrono::microseconds>(
-                      end - start).count();
+  auto duration =
+      std::chrono::duration_cast<std::chrono::microseconds>(end - start);
+
+  Stats stats(0);
+  for (const auto &w : workers) {
+    const auto &s = w->stats;
+
+    stats.req_todo += s.req_todo;
+    stats.req_started += s.req_started;
+    stats.req_done += s.req_done;
+    stats.req_success += s.req_success;
+    stats.req_status_success += s.req_status_success;
+    stats.req_failed += s.req_failed;
+    stats.req_error += s.req_error;
+    stats.bytes_total += s.bytes_total;
+    stats.bytes_head += s.bytes_head;
+    stats.bytes_body += s.bytes_body;
+
+    for (size_t i = 0; i < stats.status.size(); ++i) {
+      stats.status[i] += s.status[i];
+    }
+  }
+
+  auto time_stats = process_time_stats(workers);
 
   // Requests which have not been issued due to connection errors, are
   // counted towards req_failed and req_error.
-  auto req_not_issued = worker.stats.req_todo - worker.stats.req_success -
-                        worker.stats.req_failed;
-  worker.stats.req_failed += req_not_issued;
-  worker.stats.req_error += req_not_issued;
+  auto req_not_issued =
+      stats.req_todo - stats.req_status_success - stats.req_failed;
+  stats.req_failed += req_not_issued;
+  stats.req_error += req_not_issued;
 
-  // UI is heavily inspired by weighttp
-  // https://github.com/lighttpd/weighttp
-  size_t rps;
-  int64_t kbps;
-  if (duration > 0) {
-    auto secd = static_cast<double>(duration) / (1000 * 1000);
-    rps = worker.stats.req_todo / secd;
-    kbps = worker.stats.bytes_total / secd / 1024;
-  } else {
-    rps = 0;
-    kbps = 0;
+  // UI is heavily inspired by weighttp[1] and wrk[2]
+  //
+  // [1] https://github.com/lighttpd/weighttp
+  // [2] https://github.com/wg/wrk
+  size_t rps = 0;
+  int64_t bps = 0;
+  if (duration.count() > 0) {
+    auto secd = static_cast<double>(duration.count()) / (1000 * 1000);
+    rps = stats.req_success / secd;
+    bps = stats.bytes_total / secd;
   }
 
-  auto sec = duration / (1000 * 1000);
-  auto millisec = (duration / 1000) % 1000;
-  auto microsec = duration % 1000;
-
-  std::cout << "\n"
-            << "finished in " << sec << " sec, " << millisec << " millisec and "
-            << microsec << " microsec, " << rps << " req/s, " << kbps
-            << " kbytes/s\n"
-            << "requests: " << worker.stats.req_todo << " total, "
-            << worker.stats.req_started << " started, " << worker.stats.req_done
-            << " done, " << worker.stats.req_success << " succeeded, "
-            << worker.stats.req_failed << " failed, " << worker.stats.req_error
-            << " errored\n"
-            << "status codes: " << worker.stats.status[2] << " 2xx, "
-            << worker.stats.status[3] << " 3xx, " << worker.stats.status[4]
-            << " 4xx, " << worker.stats.status[5] << " 5xx\n"
-            << "traffic: " << worker.stats.bytes_total << " bytes total, "
-            << worker.stats.bytes_head << " bytes headers, "
-            << worker.stats.bytes_body << " bytes data" << std::endl;
+  std::cout << R"(
+finished in )" << util::format_duration(duration) << ", " << rps << " req/s, "
+            << util::utos_with_funit(bps) << R"(B/s
+requests: )" << stats.req_todo << " total, " << stats.req_started
+            << " started, " << stats.req_done << " done, "
+            << stats.req_status_success << " succeeded, " << stats.req_failed
+            << " failed, " << stats.req_error << R"( errored
+status codes: )" << stats.status[2] << " 2xx, " << stats.status[3] << " 3xx, "
+            << stats.status[4] << " 4xx, " << stats.status[5] << R"( 5xx
+traffic: )" << stats.bytes_total << " bytes total, " << stats.bytes_head
+            << " bytes headers, " << stats.bytes_body << R"( bytes data
+                     min         max         mean         sd        +/- sd
+time for request: )" << std::setw(10)
+            << util::format_duration(time_stats.time_min) << "  "
+            << std::setw(10) << util::format_duration(time_stats.time_max)
+            << "  " << std::setw(10)
+            << util::format_duration(time_stats.time_mean) << "  "
+            << std::setw(10) << util::format_duration(time_stats.time_sd)
+            << std::setw(9) << util::dtos(time_stats.within_sd) << "%"
+            << std::endl;
 
   SSL_CTX_free(ssl_ctx);
 

src/h2load.h

@@ -35,6 +35,8 @@
 #include <string>
 #include <unordered_map>
 #include <memory>
+#include <chrono>
+#include <array>
 
 #include <nghttp2/nghttp2.h>
 
@@ -43,7 +45,7 @@
 #include <openssl/ssl.h>
 
 #include "http2.h"
-#include "ringbuf.h"
+#include "buffer.h"
 
 using namespace nghttp2;
 
@@ -75,16 +77,38 @@ struct Config {
   ~Config();
 };
 
+struct RequestStat {
+  RequestStat();
+  // time point when request was sent
+  std::chrono::steady_clock::time_point request_time;
+  // time point when stream was closed
+  std::chrono::steady_clock::time_point stream_close_time;
+  // true if stream was successfully closed.  This means stream was
+  // not reset, but it does not mean HTTP level error (e.g., 404).
+  bool completed;
+};
+
+struct TimeStats {
+  // time for request: max, min, mean and sd (standard deviation)
+  std::chrono::microseconds time_max, time_min, time_mean, time_sd;
+  // percentage of number of requests inside mean -/+ sd
+  double within_sd;
+};
+
 struct Stats {
+  Stats(size_t req_todo);
   // The total number of requests
   size_t req_todo;
   // The number of requests issued so far
   size_t req_started;
   // The number of requests finished
   size_t req_done;
-  // The number of requests marked as success. This is subset of
-  // req_done.
+  // The number of requests completed successfull, but not necessarily
+  // means successful HTTP status code.
   size_t req_success;
+  // The number of requests marked as success.  HTTP status code is
+  // also considered as success. This is subset of req_done.
+  size_t req_status_success;
   // The number of requests failed. This is subset of req_done.
   size_t req_failed;
   // The number of requests failed due to network errors. This is
@@ -99,7 +123,9 @@ struct Stats {
   int64_t bytes_body;
   // The number of each HTTP status category, status[i] is status code
   // in the range [i*100, (i+1)*100).
-  size_t status[6];
+  std::array<size_t, 6> status;
+  // The statistics per request
+  std::vector<RequestStat> req_stats;
 };
 
 enum ClientState { CLIENT_IDLE, CLIENT_CONNECTED };
@@ -119,6 +145,7 @@ struct Worker {
   Worker(uint32_t id, SSL_CTX *ssl_ctx, size_t nreq_todo, size_t nclients,
          Config *config);
   ~Worker();
+  Worker(Worker &&o) = default;
   void run();
 };
 
@@ -147,7 +174,7 @@ struct Client {
   // The number of requests this client has done so far.
   size_t req_done;
   int fd;
-  RingBuf<65536> wb;
+  Buffer<65536> wb;
 
   enum { ERR_CONNECT_FAIL = -100 };
 
@@ -180,7 +207,9 @@ struct Client {
   void on_request(int32_t stream_id);
   void on_header(int32_t stream_id, const uint8_t *name, size_t namelen,
                  const uint8_t *value, size_t valuelen);
-  void on_stream_close(int32_t stream_id, bool success);
+  void on_stream_close(int32_t stream_id, bool success, RequestStat *req_stat);
+
+  void record_request_time(RequestStat *req_stat);
 
   void signal_write();
 };

src/h2load_http2_session.cc

@@ -28,6 +28,7 @@
 
 #include "h2load.h"
 #include "util.h"
+#include "template.h"
 
 using namespace nghttp2;
 
@@ -80,7 +81,31 @@ namespace {
 int on_stream_close_callback(nghttp2_session *session, int32_t stream_id,
                              uint32_t error_code, void *user_data) {
   auto client = static_cast<Client *>(user_data);
-  client->on_stream_close(stream_id, error_code == NGHTTP2_NO_ERROR);
+  auto req_stat = static_cast<RequestStat *>(
+      nghttp2_session_get_stream_user_data(session, stream_id));
+  if (!req_stat) {
+    return 0;
+  }
+  client->on_stream_close(stream_id, error_code == NGHTTP2_NO_ERROR, req_stat);
+  return 0;
+}
+} // namespace
+
+namespace {
+int before_frame_send_callback(nghttp2_session *session,
+                               const nghttp2_frame *frame, void *user_data) {
+  if (frame->hd.type != NGHTTP2_HEADERS ||
+      frame->headers.cat != NGHTTP2_HCAT_REQUEST) {
+    return 0;
+  }
+
+  auto client = static_cast<Client *>(user_data);
+  client->on_request(frame->hd.stream_id);
+  auto req_stat = static_cast<RequestStat *>(
+      nghttp2_session_get_stream_user_data(session, frame->hd.stream_id));
+  assert(req_stat);
+  client->record_request_time(req_stat);
+
   return 0;
 }
 } // namespace
@@ -106,8 +131,7 @@ void Http2Session::on_connect() {
 
   nghttp2_session_callbacks_new(&callbacks);
 
-  auto callbacks_deleter =
-      util::defer(callbacks, nghttp2_session_callbacks_del);
+  auto callbacks_deleter = defer(nghttp2_session_callbacks_del, callbacks);
 
   nghttp2_session_callbacks_set_on_frame_recv_callback(callbacks,
                                                        on_frame_recv_callback);
@@ -121,18 +145,21 @@ void Http2Session::on_connect() {
   nghttp2_session_callbacks_set_on_header_callback(callbacks,
                                                    on_header_callback);
 
+  nghttp2_session_callbacks_set_before_frame_send_callback(
+      callbacks, before_frame_send_callback);
+
   nghttp2_session_callbacks_set_send_callback(callbacks, send_callback);
 
   nghttp2_session_client_new(&session_, callbacks, client_);
 
-  nghttp2_settings_entry iv[2];
+  std::array<nghttp2_settings_entry, 2> iv;
   iv[0].settings_id = NGHTTP2_SETTINGS_ENABLE_PUSH;
   iv[0].value = 0;
   iv[1].settings_id = NGHTTP2_SETTINGS_INITIAL_WINDOW_SIZE;
   iv[1].value = (1 << client_->worker->config->window_bits) - 1;
 
-  rv = nghttp2_submit_settings(session_, NGHTTP2_FLAG_NONE, iv,
-                               util::array_size(iv));
+  rv = nghttp2_submit_settings(session_, NGHTTP2_FLAG_NONE, iv.data(),
+                               iv.size());
 
   assert(rv == 0);
 
@@ -153,7 +180,7 @@ void Http2Session::on_connect() {
   client_->signal_write();
 }
 
-void Http2Session::submit_request() {
+void Http2Session::submit_request(RequestStat *req_stat) {
   auto config = client_->worker->config;
   auto &nva = config->nva[client_->reqidx++];
 
@@ -162,10 +189,8 @@ void Http2Session::submit_request() {
   }
 
   auto stream_id = nghttp2_submit_request(session_, nullptr, nva.data(),
-                                          nva.size(), nullptr, nullptr);
+                                          nva.size(), nullptr, req_stat);
   assert(stream_id > 0);
-
-  client_->on_request(stream_id);
 }
 
 int Http2Session::on_read(const uint8_t *data, size_t len) {

src/h2load_http2_session.h

@@ -38,7 +38,7 @@ public:
   Http2Session(Client *client);
   virtual ~Http2Session();
   virtual void on_connect();
-  virtual void submit_request();
+  virtual void submit_request(RequestStat *req_stat);
   virtual int on_read(const uint8_t *data, size_t len);
   virtual int on_write();
   virtual void terminate();

src/h2load_session.h

@@ -30,6 +30,8 @@
 #include <sys/types.h>
 #include <stdint.h>
 
+#include "h2load.h"
+
 namespace h2load {
 
 class Session {
@@ -38,7 +40,7 @@ public:
   // Called when the connection was made.
   virtual void on_connect() = 0;
   // Called when one request must be issued.
-  virtual void submit_request() = 0;
+  virtual void submit_request(RequestStat *req_stat) = 0;
   // Called when incoming bytes are available. The subclass has to
   // return the number of bytes read.
   virtual int on_read(const uint8_t *data, size_t len) = 0;

src/h2load_spdy_session.cc

@@ -47,6 +47,10 @@ void before_ctrl_send_callback(spdylay_session *session,
     return;
   }
   client->on_request(frame->syn_stream.stream_id);
+  auto req_stat =
+      static_cast<RequestStat *>(spdylay_session_get_stream_user_data(
+          session, frame->syn_stream.stream_id));
+  client->record_request_time(req_stat);
 }
 } // namespace
 
@@ -86,7 +90,9 @@ void on_stream_close_callback(spdylay_session *session, int32_t stream_id,
                               spdylay_status_code status_code,
                               void *user_data) {
   auto client = static_cast<Client *>(user_data);
-  client->on_stream_close(stream_id, status_code == SPDYLAY_OK);
+  auto req_stat = static_cast<RequestStat *>(
+      spdylay_session_get_stream_user_data(session, stream_id));
+  client->on_stream_close(stream_id, status_code == SPDYLAY_OK, req_stat);
 }
 } // namespace
 
@@ -118,12 +124,11 @@ void SpdySession::on_connect() {
   spdylay_session_set_option(session_, SPDYLAY_OPT_NO_AUTO_WINDOW_UPDATE, &val,
                              sizeof(val));
 
-  spdylay_settings_entry iv[1];
-  iv[0].settings_id = SPDYLAY_SETTINGS_INITIAL_WINDOW_SIZE;
-  iv[0].flags = SPDYLAY_ID_FLAG_SETTINGS_NONE;
-  iv[0].value = (1 << client_->worker->config->window_bits);
-  spdylay_submit_settings(session_, SPDYLAY_FLAG_SETTINGS_NONE, iv,
-                          util::array_size(iv));
+  spdylay_settings_entry iv;
+  iv.settings_id = SPDYLAY_SETTINGS_INITIAL_WINDOW_SIZE;
+  iv.flags = SPDYLAY_ID_FLAG_SETTINGS_NONE;
+  iv.value = (1 << client_->worker->config->window_bits);
+  spdylay_submit_settings(session_, SPDYLAY_FLAG_SETTINGS_NONE, &iv, 1);
 
   auto config = client_->worker->config;
 
@@ -137,7 +142,7 @@ void SpdySession::on_connect() {
   client_->signal_write();
 }
 
-void SpdySession::submit_request() {
+void SpdySession::submit_request(RequestStat *req_stat) {
   auto config = client_->worker->config;
   auto &nv = config->nv[client_->reqidx++];
 
@@ -145,7 +150,7 @@ void SpdySession::submit_request() {
     client_->reqidx = 0;
   }
 
-  spdylay_submit_request(session_, 0, nv.data(), nullptr, nullptr);
+  spdylay_submit_request(session_, 0, nv.data(), nullptr, req_stat);
 }
 
 int SpdySession::on_read(const uint8_t *data, size_t len) {

src/h2load_spdy_session.h

@@ -40,7 +40,7 @@ public:
   SpdySession(Client *client, uint16_t spdy_version);
   virtual ~SpdySession();
   virtual void on_connect();
-  virtual void submit_request();
+  virtual void submit_request(RequestStat *req_stat);
   virtual int on_read(const uint8_t *data, size_t len);
   virtual int on_write();
   virtual void terminate();

src/http2.cc

@@ -165,14 +165,15 @@ void copy_url_component(std::string &dest, const http_parser_url *u, int field,
 
 Headers::value_type to_header(const uint8_t *name, size_t namelen,
                               const uint8_t *value, size_t valuelen,
-                              bool no_index) {
+                              bool no_index, int16_t token) {
   return Header(std::string(reinterpret_cast<const char *>(name), namelen),
                 std::string(reinterpret_cast<const char *>(value), valuelen),
-                no_index);
+                no_index, token);
 }
 
 void add_header(Headers &nva, const uint8_t *name, size_t namelen,
-                const uint8_t *value, size_t valuelen, bool no_index) {
+                const uint8_t *value, size_t valuelen, bool no_index,
+                int16_t token) {
   if (valuelen > 0) {
     size_t i, j;
     for (i = 0; i < valuelen && (value[i] == ' ' || value[i] == '\t'); ++i)
@@ -182,7 +183,7 @@ void add_header(Headers &nva, const uint8_t *name, size_t namelen,
     value += i;
     valuelen -= i + (valuelen - j - 1);
   }
-  nva.push_back(to_header(name, namelen, value, valuelen, no_index));
+  nva.push_back(to_header(name, namelen, value, valuelen, no_index, token));
 }
 
 const Headers::value_type *get_header(const Headers &nva, const char *name) {
@@ -221,9 +222,10 @@ void copy_headers_to_nva(std::vector<nghttp2_nv> &nva, const Headers &headers) {
     if (kv.name.empty() || kv.name[0] == ':') {
       continue;
     }
-    switch (lookup_token(kv.name)) {
+    switch (kv.token) {
     case HD_COOKIE:
     case HD_CONNECTION:
+    case HD_HOST:
     case HD_HTTP2_SETTINGS:
     case HD_KEEP_ALIVE:
     case HD_PROXY_CONNECTION:
@@ -246,9 +248,10 @@ void build_http1_headers_from_headers(std::string &hdrs,
     if (kv.name.empty() || kv.name[0] == ':') {
       continue;
     }
-    switch (lookup_token(kv.name)) {
+    switch (kv.token) {
     case HD_CONNECTION:
     case HD_COOKIE:
+    case HD_HOST:
     case HD_HTTP2_SETTINGS:
     case HD_KEEP_ALIVE:
     case HD_PROXY_CONNECTION:
@@ -324,39 +327,24 @@ void dump_nv(FILE *out, const Headers &nva) {
 
 std::string rewrite_location_uri(const std::string &uri,
                                  const http_parser_url &u,
-                                 const std::string &request_host,
-                                 const std::string &upstream_scheme,
-                                 uint16_t upstream_port) {
-  // We just rewrite host and optionally port. We don't rewrite https
-  // link. Not sure it happens in practice.
-  if (u.field_set & (1 << UF_SCHEMA)) {
-    auto field = &u.field_data[UF_SCHEMA];
-    if (!util::streq("http", &uri[field->off], field->len)) {
-      return "";
-    }
-  }
+                                 const std::string &match_host,
+                                 const std::string &request_authority,
+                                 const std::string &upstream_scheme) {
+  // We just rewrite scheme and authority.
   if ((u.field_set & (1 << UF_HOST)) == 0) {
     return "";
   }
   auto field = &u.field_data[UF_HOST];
-  if (!util::startsWith(std::begin(request_host), std::end(request_host),
+  if (!util::startsWith(std::begin(match_host), std::end(match_host),
                         &uri[field->off], &uri[field->off] + field->len) ||
-      (request_host.size() != field->len && request_host[field->len] != ':')) {
+      (match_host.size() != field->len && match_host[field->len] != ':')) {
     return "";
   }
-  std::string res = upstream_scheme;
-  res += "://";
-  res.append(&uri[field->off], field->len);
-  if (upstream_scheme == "http") {
-    if (upstream_port != 80) {
-      res += ":";
-      res += util::utos(upstream_port);
-    }
-  } else if (upstream_scheme == "https") {
-    if (upstream_port != 443) {
-      res += ":";
-      res += util::utos(upstream_port);
-    }
+  std::string res;
+  if (!request_authority.empty()) {
+    res += upstream_scheme;
+    res += "://";
+    res += request_authority;
   }
   if (u.field_set & (1 << UF_PATH)) {
     field = &u.field_data[UF_PATH];
@@ -419,7 +407,7 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 2:
     switch (name[namelen - 1]) {
     case 'e':
-      if (util::streq("t", name, 1)) {
+      if (util::streq_l("t", name, 1)) {
         return HD_TE;
       }
       break;
@@ -428,7 +416,7 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 3:
     switch (name[namelen - 1]) {
     case 'a':
-      if (util::streq("vi", name, 2)) {
+      if (util::streq_l("vi", name, 2)) {
         return HD_VIA;
       }
       break;
@@ -436,8 +424,13 @@ int lookup_token(const uint8_t *name, size_t namelen) {
     break;
   case 4:
     switch (name[namelen - 1]) {
+    case 'k':
+      if (util::streq_l("lin", name, 3)) {
+        return HD_LINK;
+      }
+      break;
     case 't':
-      if (util::streq("hos", name, 3)) {
+      if (util::streq_l("hos", name, 3)) {
         return HD_HOST;
       }
       break;
@@ -446,12 +439,12 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 5:
     switch (name[namelen - 1]) {
     case 'h':
-      if (util::streq(":pat", name, 4)) {
+      if (util::streq_l(":pat", name, 4)) {
         return HD__PATH;
       }
       break;
     case 't':
-      if (util::streq(":hos", name, 4)) {
+      if (util::streq_l(":hos", name, 4)) {
         return HD__HOST;
       }
       break;
@@ -460,17 +453,17 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 6:
     switch (name[namelen - 1]) {
     case 'e':
-      if (util::streq("cooki", name, 5)) {
+      if (util::streq_l("cooki", name, 5)) {
         return HD_COOKIE;
       }
       break;
     case 'r':
-      if (util::streq("serve", name, 5)) {
+      if (util::streq_l("serve", name, 5)) {
         return HD_SERVER;
       }
       break;
     case 't':
-      if (util::streq("expec", name, 5)) {
+      if (util::streq_l("expec", name, 5)) {
         return HD_EXPECT;
       }
       break;
@@ -479,30 +472,30 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 7:
     switch (name[namelen - 1]) {
     case 'c':
-      if (util::streq("alt-sv", name, 6)) {
+      if (util::streq_l("alt-sv", name, 6)) {
         return HD_ALT_SVC;
       }
       break;
     case 'd':
-      if (util::streq(":metho", name, 6)) {
+      if (util::streq_l(":metho", name, 6)) {
         return HD__METHOD;
       }
       break;
     case 'e':
-      if (util::streq(":schem", name, 6)) {
+      if (util::streq_l(":schem", name, 6)) {
         return HD__SCHEME;
       }
-      if (util::streq("upgrad", name, 6)) {
+      if (util::streq_l("upgrad", name, 6)) {
         return HD_UPGRADE;
       }
       break;
     case 'r':
-      if (util::streq("traile", name, 6)) {
+      if (util::streq_l("traile", name, 6)) {
         return HD_TRAILER;
       }
       break;
     case 's':
-      if (util::streq(":statu", name, 6)) {
+      if (util::streq_l(":statu", name, 6)) {
         return HD__STATUS;
       }
       break;
@@ -511,7 +504,7 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 8:
     switch (name[namelen - 1]) {
     case 'n':
-      if (util::streq("locatio", name, 7)) {
+      if (util::streq_l("locatio", name, 7)) {
         return HD_LOCATION;
       }
       break;
@@ -520,31 +513,45 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 10:
     switch (name[namelen - 1]) {
     case 'e':
-      if (util::streq("keep-aliv", name, 9)) {
+      if (util::streq_l("keep-aliv", name, 9)) {
         return HD_KEEP_ALIVE;
       }
       break;
     case 'n':
-      if (util::streq("connectio", name, 9)) {
+      if (util::streq_l("connectio", name, 9)) {
         return HD_CONNECTION;
       }
       break;
+    case 't':
+      if (util::streq_l("user-agen", name, 9)) {
+        return HD_USER_AGENT;
+      }
+      break;
     case 'y':
-      if (util::streq(":authorit", name, 9)) {
+      if (util::streq_l(":authorit", name, 9)) {
         return HD__AUTHORITY;
       }
       break;
     }
     break;
+  case 13:
+    switch (name[namelen - 1]) {
+    case 'l':
+      if (util::streq_l("cache-contro", name, 12)) {
+        return HD_CACHE_CONTROL;
+      }
+      break;
+    }
+    break;
   case 14:
     switch (name[namelen - 1]) {
     case 'h':
-      if (util::streq("content-lengt", name, 13)) {
+      if (util::streq_l("content-lengt", name, 13)) {
         return HD_CONTENT_LENGTH;
       }
       break;
     case 's':
-      if (util::streq("http2-setting", name, 13)) {
+      if (util::streq_l("http2-setting", name, 13)) {
         return HD_HTTP2_SETTINGS;
       }
       break;
@@ -552,8 +559,18 @@ int lookup_token(const uint8_t *name, size_t namelen) {
     break;
   case 15:
     switch (name[namelen - 1]) {
+    case 'e':
+      if (util::streq_l("accept-languag", name, 14)) {
+        return HD_ACCEPT_LANGUAGE;
+      }
+      break;
+    case 'g':
+      if (util::streq_l("accept-encodin", name, 14)) {
+        return HD_ACCEPT_ENCODING;
+      }
+      break;
     case 'r':
-      if (util::streq("x-forwarded-fo", name, 14)) {
+      if (util::streq_l("x-forwarded-fo", name, 14)) {
         return HD_X_FORWARDED_FOR;
       }
       break;
@@ -562,7 +579,7 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 16:
     switch (name[namelen - 1]) {
     case 'n':
-      if (util::streq("proxy-connectio", name, 15)) {
+      if (util::streq_l("proxy-connectio", name, 15)) {
         return HD_PROXY_CONNECTION;
       }
       break;
@@ -571,17 +588,17 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   case 17:
     switch (name[namelen - 1]) {
     case 'e':
-      if (util::streq("if-modified-sinc", name, 16)) {
+      if (util::streq_l("if-modified-sinc", name, 16)) {
         return HD_IF_MODIFIED_SINCE;
       }
       break;
     case 'g':
-      if (util::streq("transfer-encodin", name, 16)) {
+      if (util::streq_l("transfer-encodin", name, 16)) {
         return HD_TRANSFER_ENCODING;
       }
       break;
     case 'o':
-      if (util::streq("x-forwarded-prot", name, 16)) {
+      if (util::streq_l("x-forwarded-prot", name, 16)) {
         return HD_X_FORWARDED_PROTO;
       }
       break;
@@ -591,20 +608,11 @@ int lookup_token(const uint8_t *name, size_t namelen) {
   return -1;
 }
 
-void init_hdidx(int *hdidx) { memset(hdidx, -1, sizeof(hdidx[0]) * HD_MAXIDX); }
-
-void index_headers(int *hdidx, const Headers &headers) {
-  for (size_t i = 0; i < headers.size(); ++i) {
-    auto &kv = headers[i];
-    auto token = lookup_token(
-        reinterpret_cast<const uint8_t *>(kv.name.c_str()), kv.name.size());
-    if (token >= 0) {
-      http2::index_header(hdidx, token, i);
-    }
-  }
+void init_hdidx(HeaderIndex &hdidx) {
+  std::fill(std::begin(hdidx), std::end(hdidx), -1);
 }
 
-void index_header(int *hdidx, int token, size_t idx) {
+void index_header(HeaderIndex &hdidx, int16_t token, size_t idx) {
   if (token == -1) {
     return;
   }
@@ -612,7 +620,8 @@ void index_header(int *hdidx, int token, size_t idx) {
   hdidx[token] = idx;
 }
 
-bool check_http2_request_pseudo_header(const int *hdidx, int token) {
+bool check_http2_request_pseudo_header(const HeaderIndex &hdidx,
+                                       int16_t token) {
   switch (token) {
   case HD__AUTHORITY:
   case HD__METHOD:
@@ -624,7 +633,8 @@ bool check_http2_request_pseudo_header(const int *hdidx, int token) {
   }
 }
 
-bool check_http2_response_pseudo_header(const int *hdidx, int token) {
+bool check_http2_response_pseudo_header(const HeaderIndex &hdidx,
+                                        int16_t token) {
   switch (token) {
   case HD__STATUS:
     return hdidx[token] == -1;
@@ -633,7 +643,7 @@ bool check_http2_response_pseudo_header(const int *hdidx, int token) {
   }
 }
 
-bool http2_header_allowed(int token) {
+bool http2_header_allowed(int16_t token) {
   switch (token) {
   case HD_CONNECTION:
   case HD_KEEP_ALIVE:
@@ -646,7 +656,7 @@ bool http2_header_allowed(int token) {
   }
 }
 
-bool http2_mandatory_request_headers_presence(const int *hdidx) {
+bool http2_mandatory_request_headers_presence(const HeaderIndex &hdidx) {
   if (hdidx[HD__METHOD] == -1 || hdidx[HD__PATH] == -1 ||
       hdidx[HD__SCHEME] == -1 ||
       (hdidx[HD__AUTHORITY] == -1 && hdidx[HD_HOST] == -1)) {
@@ -655,7 +665,7 @@ bool http2_mandatory_request_headers_presence(const int *hdidx) {
   return true;
 }
 
-const Headers::value_type *get_header(const int *hdidx, int token,
+const Headers::value_type *get_header(const HeaderIndex &hdidx, int16_t token,
                                       const Headers &nva) {
   auto i = hdidx[token];
   if (i == -1) {
@@ -664,36 +674,422 @@ const Headers::value_type *get_header(const int *hdidx, int token,
   return &nva[i];
 }
 
-bool check_http2_te(const uint8_t *value, size_t valuelen) {
-  auto first = value;
-  auto last = first + valuelen;
-  for (;;) {
-    if (first == last) {
-      return true;
-    }
-    auto end_param = std::find(first, last, ',');
-    auto len = end_param - first;
-    if (!util::istartsWith(reinterpret_cast<const char *>(first), len,
-                           "trailer")) {
-      return false;
-    }
-    if (len == sizeof("trailer") - 1) {
-      goto next;
-    }
-    switch (first[sizeof("trailer") - 1]) {
+namespace {
+template <typename InputIt> InputIt skip_lws(InputIt first, InputIt last) {
+  for (; first != last; ++first) {
+    switch (*first) {
     case ' ':
-    case ';':
-      goto next;
+    case '\t':
+      continue;
+    default:
+      return first;
     }
-    return false;
-  next:
-    if (end_param == last) {
-      return true;
-    }
-    first = std::find_if_not(end_param + 1, last, [](uint8_t c) {
-      return c == ' ' || c == '\t' || c == ',';
-    });
   }
+  return first;
+}
+} // namespace
+
+namespace {
+template <typename InputIt>
+InputIt skip_to_next_field(InputIt first, InputIt last) {
+  for (; first != last; ++first) {
+    switch (*first) {
+    case ' ':
+    case '\t':
+    case ',':
+      continue;
+    default:
+      return first;
+    }
+  }
+  return first;
+}
+} // namespace
+
+namespace {
+// Skip to the right dquote ('"'), handling backslash escapes.
+// Returns |last| if input is not terminated with '"'.
+template <typename InputIt>
+InputIt skip_to_right_dquote(InputIt first, InputIt last) {
+  for (; first != last;) {
+    switch (*first) {
+    case '"':
+      return first;
+    case '\\':
+      ++first;
+      if (first == last) {
+        return first;
+      }
+      break;
+    }
+    ++first;
+  }
+  return first;
+}
+} // namespace
+
+namespace {
+std::pair<LinkHeader, const char *>
+parse_next_link_header_once(const char *first, const char *last) {
+  first = skip_to_next_field(first, last);
+  if (first == last || *first != '<') {
+    return {{{nullptr, nullptr}}, last};
+  }
+  auto url_first = ++first;
+  first = std::find(first, last, '>');
+  if (first == last) {
+    return {{{nullptr, nullptr}}, first};
+  }
+  auto url_last = first++;
+  if (first == last) {
+    return {{{nullptr, nullptr}}, first};
+  }
+  // we expect ';' or ',' here
+  switch (*first) {
+  case ',':
+    return {{{nullptr, nullptr}}, ++first};
+  case ';':
+    ++first;
+    break;
+  default:
+    return {{{nullptr, nullptr}}, last};
+  }
+
+  auto ok = false;
+  auto ign = false;
+  for (;;) {
+    first = skip_lws(first, last);
+    if (first == last) {
+      return {{{nullptr, nullptr}}, first};
+    }
+    // we expect link-param
+
+    // rel can take several relations using quoted form.
+    static const char PLP[] = "rel=\"";
+    static const size_t PLPLEN = sizeof(PLP) - 1;
+
+    static const char PLT[] = "preload";
+    static const size_t PLTLEN = sizeof(PLT) - 1;
+    if (first + PLPLEN < last && *(first + PLPLEN - 1) == '"' &&
+        std::equal(PLP, PLP + PLPLEN, first, util::CaseCmp())) {
+      // we have to search preload in whitespace separated list:
+      // rel="preload something http://example.org/foo"
+      first += PLPLEN;
+      auto start = first;
+      for (; first != last;) {
+        if (*first != ' ' && *first != '"') {
+          ++first;
+          continue;
+        }
+
+        if (start == first) {
+          return {{{nullptr, nullptr}}, last};
+        }
+
+        if (!ok && start + PLTLEN == first &&
+            std::equal(PLT, PLT + PLTLEN, start, util::CaseCmp())) {
+          ok = true;
+        }
+
+        if (*first == '"') {
+          break;
+        }
+        first = skip_lws(first, last);
+        start = first;
+      }
+      if (first == last) {
+        return {{{nullptr, nullptr}}, first};
+      }
+      assert(*first == '"');
+      ++first;
+      if (first == last || *first == ',') {
+        goto almost_done;
+      }
+      if (*first == ';') {
+        ++first;
+        // parse next link-param
+        continue;
+      }
+      return {{{nullptr, nullptr}}, last};
+    }
+    // we are only interested in rel=preload parameter.  Others are
+    // simply skipped.
+    static const char PL[] = "rel=preload";
+    static const size_t PLLEN = sizeof(PL) - 1;
+    if (first + PLLEN == last) {
+      if (std::equal(PL, PL + PLLEN, first, util::CaseCmp())) {
+        ok = true;
+        // this is the end of sequence
+        return {{{url_first, url_last}}, last};
+      }
+    } else if (first + PLLEN + 1 <= last) {
+      switch (*(first + PLLEN)) {
+      case ',':
+        if (!std::equal(PL, PL + PLLEN, first, util::CaseCmp())) {
+          break;
+        }
+        ok = true;
+        // skip including ','
+        first += PLLEN + 1;
+        return {{{url_first, url_last}}, first};
+      case ';':
+        if (!std::equal(PL, PL + PLLEN, first, util::CaseCmp())) {
+          break;
+        }
+        ok = true;
+        // skip including ';'
+        first += PLLEN + 1;
+        // continue parse next link-param
+        continue;
+      }
+    }
+    // we have to reject URI if we have nonempty anchor parameter.
+    static const char ANCHOR[] = "anchor=";
+    static const size_t ANCHORLEN = sizeof(ANCHOR) - 1;
+    if (!ign && first + ANCHORLEN <= last) {
+      if (std::equal(ANCHOR, ANCHOR + ANCHORLEN, first, util::CaseCmp())) {
+        // we only accept URI if anchor="" here.
+        if (first + ANCHORLEN + 2 <= last) {
+          if (*(first + ANCHORLEN) != '"' || *(first + ANCHORLEN + 1) != '"') {
+            ign = true;
+          }
+        } else {
+          // here we got invalid production (anchor=") or anchor=?
+          ign = true;
+        }
+      }
+    }
+
+    auto param_first = first;
+    for (; first != last;) {
+      if (util::in_attr_char(*first)) {
+        ++first;
+        continue;
+      }
+      // '*' is only allowed at the end of parameter name and must be
+      // followed by '='
+      if (last - first >= 2 && first != param_first) {
+        if (*first == '*' && *(first + 1) == '=') {
+          ++first;
+          break;
+        }
+      }
+      if (*first == '=' || *first == ';' || *first == ',') {
+        break;
+      }
+      return {{{nullptr, nullptr}}, last};
+    }
+    if (param_first == first) {
+      // empty parmname
+      return {{{nullptr, nullptr}}, last};
+    }
+    // link-param without value is acceptable (see link-extension) if
+    // it is not followed by '='
+    if (first == last || *first == ',') {
+      goto almost_done;
+    }
+    if (*first == ';') {
+      ++first;
+      // parse next link-param
+      continue;
+    }
+    // now parsing link-param value
+    assert(*first == '=');
+    ++first;
+    if (first == last) {
+      // empty value is not acceptable
+      return {{{nullptr, nullptr}}, first};
+    }
+    if (*first == '"') {
+      // quoted-string
+      first = skip_to_right_dquote(first + 1, last);
+      if (first == last) {
+        return {{{nullptr, nullptr}}, first};
+      }
+      ++first;
+      if (first == last || *first == ',') {
+        goto almost_done;
+      }
+      if (*first == ';') {
+        ++first;
+        // parse next link-param
+        continue;
+      }
+      return {{{nullptr, nullptr}}, last};
+    }
+    // not quoted-string, skip to next ',' or ';'
+    if (*first == ',' || *first == ';') {
+      // empty value
+      return {{{nullptr, nullptr}}, last};
+    }
+    for (; first != last; ++first) {
+      if (*first == ',' || *first == ';') {
+        break;
+      }
+    }
+    if (first == last || *first == ',') {
+      goto almost_done;
+    }
+    assert(*first == ';');
+    ++first;
+    // parse next link-param
+  }
+
+almost_done:
+  assert(first == last || *first == ',');
+
+  if (first != last) {
+    ++first;
+  }
+  if (ok && !ign) {
+    return {{{url_first, url_last}}, first};
+  }
+  return {{{nullptr, nullptr}}, first};
+}
+} // namespace
+
+std::vector<LinkHeader> parse_link_header(const char *src, size_t len) {
+  auto first = src;
+  auto last = src + len;
+  std::vector<LinkHeader> res;
+  for (; first != last;) {
+    auto rv = parse_next_link_header_once(first, last);
+    first = rv.second;
+    if (rv.first.uri.first != nullptr && rv.first.uri.second != nullptr) {
+      res.push_back(rv.first);
+    }
+  }
+  return res;
+}
+
+namespace {
+void eat_file(std::string &path) {
+  if (path.empty()) {
+    path = "/";
+    return;
+  }
+  auto p = path.size() - 1;
+  if (path[p] == '/') {
+    return;
+  }
+  p = path.rfind('/', p);
+  if (p == std::string::npos) {
+    // this should not happend in normal case, where we expect path
+    // starts with '/'
+    path = "/";
+    return;
+  }
+  path.erase(std::begin(path) + p + 1, std::end(path));
+}
+} // namespace
+
+namespace {
+void eat_dir(std::string &path) {
+  if (path.empty()) {
+    path = "/";
+    return;
+  }
+  auto p = path.size() - 1;
+  if (path[p] != '/') {
+    p = path.rfind('/', p);
+    if (p == std::string::npos) {
+      // this should not happend in normal case, where we expect path
+      // starts with '/'
+      path = "/";
+      return;
+    }
+  }
+  if (path[p] == '/') {
+    if (p == 0) {
+      return;
+    }
+    --p;
+  }
+  p = path.rfind('/', p);
+  if (p == std::string::npos) {
+    // this should not happend in normal case, where we expect path
+    // starts with '/'
+    path = "/";
+    return;
+  }
+  path.erase(std::begin(path) + p + 1, std::end(path));
+}
+} // namespace
+
+std::string path_join(const char *base_path, size_t base_pathlen,
+                      const char *base_query, size_t base_querylen,
+                      const char *rel_path, size_t rel_pathlen,
+                      const char *rel_query, size_t rel_querylen) {
+  std::string res;
+  if (rel_pathlen == 0) {
+    if (base_pathlen == 0) {
+      res = "/";
+    } else {
+      res.assign(base_path, base_pathlen);
+    }
+    if (rel_querylen == 0) {
+      if (base_querylen) {
+        res += "?";
+        res.append(base_query, base_querylen);
+      }
+      return res;
+    }
+    res += "?";
+    res.append(rel_query, rel_querylen);
+    return res;
+  }
+
+  auto first = rel_path;
+  auto last = rel_path + rel_pathlen;
+
+  if (rel_path[0] == '/') {
+    res = "/";
+    ++first;
+  } else if (base_pathlen == 0) {
+    res = "/";
+  } else {
+    res.assign(base_path, base_pathlen);
+  }
+
+  for (; first != last;) {
+    if (*first == '.') {
+      if (first + 1 == last) {
+        break;
+      }
+      if (*(first + 1) == '/') {
+        first += 2;
+        continue;
+      }
+      if (*(first + 1) == '.') {
+        if (first + 2 == last) {
+          eat_dir(res);
+          break;
+        }
+        if (*(first + 2) == '/') {
+          eat_dir(res);
+          first += 3;
+          continue;
+        }
+      }
+    }
+    if (res.back() != '/') {
+      eat_file(res);
+    }
+    auto slash = std::find(first, last, '/');
+    if (slash == last) {
+      res.append(first, last);
+      break;
+    }
+    res.append(first, slash + 1);
+    first = slash + 1;
+    for (; first != last && *first == '/'; ++first)
+      ;
+  }
+  if (rel_querylen) {
+    res += "?";
+    res.append(rel_query, rel_querylen);
+  }
+  return res;
 }
 
 } // namespace http2

src/http2.h

@@ -31,6 +31,7 @@
 #include <cstring>
 #include <string>
 #include <vector>
+#include <array>
 
 #include <nghttp2/nghttp2.h>
 
@@ -39,10 +40,12 @@
 namespace nghttp2 {
 
 struct Header {
-  Header(std::string name, std::string value, bool no_index = false)
-      : name(std::move(name)), value(std::move(value)), no_index(no_index) {}
+  Header(std::string name, std::string value, bool no_index = false,
+         int16_t token = -1)
+      : name(std::move(name)), value(std::move(value)), token(token),
+        no_index(no_index) {}
 
-  Header() : no_index(false) {}
+  Header() : token(-1), no_index(false) {}
 
   bool operator==(const Header &other) const {
     return name == other.name && value == other.value;
@@ -54,6 +57,7 @@ struct Header {
 
   std::string name;
   std::string value;
+  int16_t token;
   bool no_index;
 };
 
@@ -76,13 +80,14 @@ void copy_url_component(std::string &dest, const http_parser_url *u, int field,
 
 Headers::value_type to_header(const uint8_t *name, size_t namelen,
                               const uint8_t *value, size_t valuelen,
-                              bool no_index);
+                              bool no_index, int16_t token);
 
 // Add name/value pairs to |nva|.  If |no_index| is true, this
 // name/value pair won't be indexed when it is forwarded to the next
 // hop.  This function strips white spaces around |value|.
 void add_header(Headers &nva, const uint8_t *name, size_t namelen,
-                const uint8_t *value, size_t valuelen, bool no_index);
+                const uint8_t *value, size_t valuelen, bool no_index,
+                int16_t token);
 
 // Returns pointer to the entry in |nva| which has name |name|.  If
 // more than one entries which have the name |name|, last occurrence
@@ -124,14 +129,16 @@ nghttp2_nv make_nv_ls(const char (&name)[N], const std::string &value) {
           NGHTTP2_NV_FLAG_NONE};
 }
 
-// Appends headers in |headers| to |nv|. Certain headers, including
-// disallowed headers in HTTP/2 spec and headers which require
-// special handling (i.e. via), are not copied.
+// Appends headers in |headers| to |nv|.  |headers| must be indexed
+// before this call (its element's token field is assigned).  Certain
+// headers, including disallowed headers in HTTP/2 spec and headers
+// which require special handling (i.e. via), are not copied.
 void copy_headers_to_nva(std::vector<nghttp2_nv> &nva, const Headers &headers);
 
 // Appends HTTP/1.1 style header lines to |hdrs| from headers in
-// |headers|. Certain headers, which requires special handling
-// (i.e. via and cookie), are not appended.
+// |headers|.  |headers| must be indexed before this call (its
+// element's token field is assigned).  Certain headers, which
+// requires special handling (i.e. via and cookie), are not appended.
 void build_http1_headers_from_headers(std::string &hdrs,
                                       const Headers &headers);
 
@@ -156,19 +163,22 @@ void dump_nv(FILE *out, const Headers &nva);
 
 // Rewrites redirection URI which usually appears in location header
 // field. The |uri| is the URI in the location header field. The |u|
-// stores the result of parsed |uri|. The |request_host| is the host
-// or :authority header field value in the request. The
+// stores the result of parsed |uri|. The |request_authority| is the
+// host or :authority header field value in the request. The
 // |upstream_scheme| is either "https" or "http" in the upstream
-// interface.
+// interface.  Rewrite is done only if location header field value
+// contains |match_host| as host excluding port.  The |match_host| and
+// |request_authority| could be different.  If |request_authority| is
+// empty, strip authority.
 //
 // This function returns the new rewritten URI on success. If the
 // location URI is not subject to the rewrite, this function returns
 // emtpy string.
 std::string rewrite_location_uri(const std::string &uri,
                                  const http_parser_url &u,
-                                 const std::string &request_host,
-                                 const std::string &upstream_scheme,
-                                 uint16_t upstream_port);
+                                 const std::string &match_host,
+                                 const std::string &request_authority,
+                                 const std::string &upstream_scheme);
 
 // Checks the header name/value pair using nghttp2_check_header_name()
 // and nghttp2_check_header_value(). If both function returns nonzero,
@@ -188,7 +198,10 @@ enum {
   HD__PATH,
   HD__SCHEME,
   HD__STATUS,
+  HD_ACCEPT_ENCODING,
+  HD_ACCEPT_LANGUAGE,
   HD_ALT_SVC,
+  HD_CACHE_CONTROL,
   HD_CONNECTION,
   HD_CONTENT_LENGTH,
   HD_COOKIE,
@@ -197,6 +210,7 @@ enum {
   HD_HTTP2_SETTINGS,
   HD_IF_MODIFIED_SINCE,
   HD_KEEP_ALIVE,
+  HD_LINK,
   HD_LOCATION,
   HD_PROXY_CONNECTION,
   HD_SERVER,
@@ -204,12 +218,15 @@ enum {
   HD_TRAILER,
   HD_TRANSFER_ENCODING,
   HD_UPGRADE,
+  HD_USER_AGENT,
   HD_VIA,
   HD_X_FORWARDED_FOR,
   HD_X_FORWARDED_PROTO,
   HD_MAXIDX,
 };
 
+using HeaderIndex = std::array<int16_t, HD_MAXIDX>;
+
 // Looks up header token for header name |name| of length |namelen|.
 // Only headers we are interested in are tokenized.  If header name
 // cannot be tokenized, returns -1.
@@ -218,37 +235,52 @@ int lookup_token(const std::string &name);
 
 // Initializes |hdidx|, header index.  The |hdidx| must point to the
 // array containing at least HD_MAXIDX elements.
-void init_hdidx(int *hdidx);
+void init_hdidx(HeaderIndex &hdidx);
 // Indexes header |token| using index |idx|.
-void index_header(int *hdidx, int token, size_t idx);
-// Iterates |headers| and for each element, call index_header.
-void index_headers(int *hdidx, const Headers &headers);
+void index_header(HeaderIndex &hdidx, int16_t token, size_t idx);
 
 // Returns true if HTTP/2 request pseudo header |token| is not indexed
 // yet and not -1.
-bool check_http2_request_pseudo_header(const int *hdidx, int token);
+bool check_http2_request_pseudo_header(const HeaderIndex &hdidx, int16_t token);
 
 // Returns true if HTTP/2 response pseudo header |token| is not
 // indexed yet and not -1.
-bool check_http2_response_pseudo_header(const int *hdidx, int token);
+bool check_http2_response_pseudo_header(const HeaderIndex &hdidx,
+                                        int16_t token);
 
 // Returns true if header field denoted by |token| is allowed for
 // HTTP/2.
-bool http2_header_allowed(int token);
+bool http2_header_allowed(int16_t token);
 
 // Returns true that |hdidx| contains mandatory HTTP/2 request
 // headers.
-bool http2_mandatory_request_headers_presence(const int *hdidx);
+bool http2_mandatory_request_headers_presence(const HeaderIndex &hdidx);
 
 // Returns header denoted by |token| using index |hdidx|.
-const Headers::value_type *get_header(const int *hdidx, int token,
+const Headers::value_type *get_header(const HeaderIndex &hdidx, int16_t token,
                                       const Headers &nva);
 
-// Returns true if TE request header field value |value| of length
-// |valuelen| is empty or only contains "trailers".  The valid header
-// field value is "trailer" and optionally followed by "," or OWS for
-// simplicity.
-bool check_http2_te(const uint8_t *value, size_t valuelen);
+struct LinkHeader {
+  // The region of URI is [uri.first, uri.second).
+  std::pair<const char *, const char *> uri;
+};
+
+// Returns next URI-reference in Link header field value |src| of
+// length |len|.  If no URI-reference found after searching all input,
+// returned uri field is empty.  This imply that empty URI-reference
+// is ignored during parsing.
+std::vector<LinkHeader> parse_link_header(const char *src, size_t len);
+
+// Constructs path by combining base path |base_path| of length
+// |base_pathlen| with another path |rel_path| of length
+// |rel_pathlen|.  The base path and another path can have optional
+// query component.  This function assumes |base_path| is
+// cannibalized.  In other words, it does not contain ".." or "." path
+// components and starts with "/" if it is not empty.
+std::string path_join(const char *base_path, size_t base_pathlen,
+                      const char *base_query, size_t base_querylen,
+                      const char *rel_path, size_t rel_pathlen,
+                      const char *rel_query, size_t rel_querylen);
 
 } // namespace http2
 

src/http2_test.cc

@@ -58,46 +58,52 @@ void test_http2_add_header(void) {
   auto nva = Headers();
 
   http2::add_header(nva, (const uint8_t *)"alpha", 5, (const uint8_t *)"123", 3,
-                    false);
+                    false, -1);
   CU_ASSERT(Headers::value_type("alpha", "123") == nva[0]);
   CU_ASSERT(!nva[0].no_index);
 
   nva.clear();
 
   http2::add_header(nva, (const uint8_t *)"alpha", 5, (const uint8_t *)"", 0,
-                    true);
+                    true, -1);
   CU_ASSERT(Headers::value_type("alpha", "") == nva[0]);
   CU_ASSERT(nva[0].no_index);
 
   nva.clear();
 
   http2::add_header(nva, (const uint8_t *)"a", 1, (const uint8_t *)" b", 2,
-                    false);
+                    false, -1);
   CU_ASSERT(Headers::value_type("a", "b") == nva[0]);
 
   nva.clear();
 
   http2::add_header(nva, (const uint8_t *)"a", 1, (const uint8_t *)"b ", 2,
-                    false);
+                    false, -1);
   CU_ASSERT(Headers::value_type("a", "b") == nva[0]);
 
   nva.clear();
 
   http2::add_header(nva, (const uint8_t *)"a", 1, (const uint8_t *)"  b  ", 5,
-                    false);
+                    false, -1);
   CU_ASSERT(Headers::value_type("a", "b") == nva[0]);
 
   nva.clear();
 
   http2::add_header(nva, (const uint8_t *)"a", 1, (const uint8_t *)"  bravo  ",
-                    9, false);
+                    9, false, -1);
   CU_ASSERT(Headers::value_type("a", "bravo") == nva[0]);
 
   nva.clear();
 
   http2::add_header(nva, (const uint8_t *)"a", 1, (const uint8_t *)"    ", 4,
-                    false);
+                    false, -1);
   CU_ASSERT(Headers::value_type("a", "") == nva[0]);
+
+  nva.clear();
+
+  http2::add_header(nva, (const uint8_t *)"te", 2, (const uint8_t *)"trailers",
+                    8, false, http2::HD_TE);
+  CU_ASSERT(http2::HD_TE == nva[0].token);
 }
 
 void test_http2_get_header(void) {
@@ -120,7 +126,7 @@ void test_http2_get_header(void) {
   rv = http2::get_header(nva, "foxtrot");
   CU_ASSERT(rv == nullptr);
 
-  int hdidx[http2::HD_MAXIDX];
+  http2::HeaderIndex hdidx;
   http2::init_hdidx(hdidx);
   hdidx[http2::HD_CONTENT_LENGTH] = 6;
   rv = http2::get_header(hdidx, http2::HD_CONTENT_LENGTH, nva);
@@ -128,19 +134,20 @@ void test_http2_get_header(void) {
 }
 
 namespace {
-auto headers = Headers{{"alpha", "0", true},
-                       {"bravo", "1"},
-                       {"connection", "2"},
-                       {"connection", "3"},
-                       {"delta", "4"},
-                       {"expect", "5"},
-                       {"foxtrot", "6"},
-                       {"tango", "7"},
-                       {"te", "8"},
-                       {"te", "9"},
-                       {"x-forwarded-proto", "10"},
-                       {"x-forwarded-proto", "11"},
-                       {"zulu", "12"}};
+auto headers =
+    Headers{{"alpha", "0", true},
+            {"bravo", "1"},
+            {"connection", "2", false, http2::HD_CONNECTION},
+            {"connection", "3", false, http2::HD_CONNECTION},
+            {"delta", "4"},
+            {"expect", "5"},
+            {"foxtrot", "6"},
+            {"tango", "7"},
+            {"te", "8", false, http2::HD_TE},
+            {"te", "9", false, http2::HD_TE},
+            {"x-forwarded-proto", "10", false, http2::HD_X_FORWARDED_FOR},
+            {"x-forwarded-proto", "11", false, http2::HD_X_FORWARDED_FOR},
+            {"zulu", "12"}};
 } // namespace
 
 void test_http2_copy_headers_to_nva(void) {
@@ -180,40 +187,44 @@ void test_http2_lws(void) {
 }
 
 namespace {
-void check_rewrite_location_uri(const std::string &new_uri,
-                                const std::string &uri,
-                                const std::string &req_host,
-                                const std::string &upstream_scheme,
-                                uint16_t upstream_port) {
+void check_rewrite_location_uri(const std::string &want, const std::string &uri,
+                                const std::string &match_host,
+                                const std::string &req_authority,
+                                const std::string &upstream_scheme) {
   http_parser_url u;
   memset(&u, 0, sizeof(u));
   CU_ASSERT(0 == http_parser_parse_url(uri.c_str(), uri.size(), 0, &u));
-  CU_ASSERT(new_uri == http2::rewrite_location_uri(
-                           uri, u, req_host, upstream_scheme, upstream_port));
+  auto got = http2::rewrite_location_uri(uri, u, match_host, req_authority,
+                                         upstream_scheme);
+  CU_ASSERT(want == got);
 }
 } // namespace
 
 void test_http2_rewrite_location_uri(void) {
   check_rewrite_location_uri("https://localhost:3000/alpha?bravo#charlie",
                              "http://localhost:3001/alpha?bravo#charlie",
-                             "localhost:3001", "https", 3000);
+                             "localhost:3001", "localhost:3000", "https");
   check_rewrite_location_uri("https://localhost/", "http://localhost:3001/",
-                             "localhost:3001", "https", 443);
+                             "localhost", "localhost", "https");
   check_rewrite_location_uri("http://localhost/", "http://localhost:3001/",
-                             "localhost:3001", "http", 80);
+                             "localhost", "localhost", "http");
   check_rewrite_location_uri("http://localhost:443/", "http://localhost:3001/",
-                             "localhost:3001", "http", 443);
+                             "localhost", "localhost:443", "http");
   check_rewrite_location_uri("https://localhost:80/", "http://localhost:3001/",
-                             "localhost:3001", "https", 80);
-  check_rewrite_location_uri("", "http://localhost:3001/", "127.0.0.1", "https",
-                             3000);
+                             "localhost", "localhost:80", "https");
+  check_rewrite_location_uri("", "http://localhost:3001/", "127.0.0.1",
+                             "127.0.0.1", "https");
   check_rewrite_location_uri("https://localhost:3000/",
-                             "http://localhost:3001/", "localhost", "https",
-                             3000);
-  check_rewrite_location_uri("", "https://localhost:3001/", "localhost",
-                             "https", 3000);
+                             "http://localhost:3001/", "localhost",
+                             "localhost:3000", "https");
   check_rewrite_location_uri("https://localhost:3000/", "http://localhost/",
-                             "localhost", "https", 3000);
+                             "localhost", "localhost:3000", "https");
+
+  // match_host != req_authority
+  check_rewrite_location_uri("https://example.org", "http://127.0.0.1:8080",
+                             "127.0.0.1", "example.org", "https");
+  check_rewrite_location_uri("", "http://example.org", "127.0.0.1",
+                             "example.org", "https");
 }
 
 void test_http2_parse_http_status_code(void) {
@@ -227,7 +238,7 @@ void test_http2_parse_http_status_code(void) {
 }
 
 void test_http2_index_header(void) {
-  int hdidx[http2::HD_MAXIDX];
+  http2::HeaderIndex hdidx;
   http2::init_hdidx(hdidx);
 
   http2::index_header(hdidx, http2::HD__AUTHORITY, 0);
@@ -244,7 +255,7 @@ void test_http2_lookup_token(void) {
 }
 
 void test_http2_check_http2_pseudo_header(void) {
-  int hdidx[http2::HD_MAXIDX];
+  http2::HeaderIndex hdidx;
   http2::init_hdidx(hdidx);
 
   CU_ASSERT(http2::check_http2_request_pseudo_header(hdidx, http2::HD__METHOD));
@@ -272,7 +283,7 @@ void test_http2_http2_header_allowed(void) {
 }
 
 void test_http2_mandatory_request_headers_presence(void) {
-  int hdidx[http2::HD_MAXIDX];
+  http2::HeaderIndex hdidx;
   http2::init_hdidx(hdidx);
 
   CU_ASSERT(!http2::http2_mandatory_request_headers_presence(hdidx));
@@ -290,59 +301,512 @@ void test_http2_mandatory_request_headers_presence(void) {
   CU_ASSERT(http2::http2_mandatory_request_headers_presence(hdidx));
 }
 
-void test_http2_check_http2_te(void) {
+void test_http2_parse_link_header(void) {
   {
-    const uint8_t v[] = "trailer";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // only URI appears; we don't extract URI unless it bears rel=preload
+    const char s[] = "<url>";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
   }
   {
-    const uint8_t v[] = "Trailer";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // URI url should be extracted
+    const char s[] = "<url>; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
   }
   {
-    const uint8_t v[] = "trailer,";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // With extra link-param.  URI url should be extracted
+    const char s[] = "<url>; rel=preload; as=file";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
   }
   {
-    const uint8_t v[] = "trailer,,";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // With extra link-param.  URI url should be extracted
+    const char s[] = "<url>; as=file; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
   }
   {
-    const uint8_t v[] = "trailer, ,";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // With extra link-param and quote-string.  URI url should be
+    // extracted
+    const char s[] = R"(<url>; rel=preload; title="foo,bar")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
   }
   {
-    const uint8_t v[] = "trailer, , ";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // With extra link-param and quote-string.  URI url should be
+    // extracted
+    const char s[] = R"(<url>; title="foo,bar"; rel=preload)";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
   }
   {
-    const uint8_t v[] = "trailer; q=0.9";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // ',' after quote-string
+    const char s[] = R"(<url>; title="foo,bar", <url>; rel=preload)";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[25], &s[28]) == res[0].uri);
   }
   {
-    const uint8_t v[] = "trailer ; q=0.9";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // Only first URI should be extracted.
+    const char s[] = "<url>; rel=preload, <url>";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
   }
   {
-    const uint8_t v[] = "trailer ; q=0.9, trailer";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
+    // Both have rel=preload, so both urls should be extracted
+    const char s[] = "<url>; rel=preload, <url>; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(2 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+    CU_ASSERT(std::make_pair(&s[21], &s[24]) == res[1].uri);
   }
   {
-    const uint8_t v[] = "trailer ; q=0.9,  trailer";
-    CU_ASSERT(http2::check_http2_te(v, sizeof(v) - 1));
-  }
-  // failure cases
-  {
-    const uint8_t v[] = "trailer; q=0.9, gzip";
-    CU_ASSERT(!http2::check_http2_te(v, sizeof(v) - 1));
+    // Second URI uri should be extracted.
+    const char s[] = "<url>, <url>;rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[8], &s[11]) == res[0].uri);
   }
   {
-    const uint8_t v[] = "traile";
-    CU_ASSERT(!http2::check_http2_te(v, sizeof(v) - 1));
+    // Error if input ends with ';'
+    const char s[] = "<url>;rel=preload;";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
   }
   {
-    const uint8_t v[] = "trailerr";
-    CU_ASSERT(!http2::check_http2_te(v, sizeof(v) - 1));
+    // OK if input ends with ','
+    const char s[] = "<url>;rel=preload,";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // Multiple repeated ','s between fields is OK
+    const char s[] = "<url>,,,<url>;rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[9], &s[12]) == res[0].uri);
+  }
+  {
+    // Error if url is not enclosed by <>
+    const char s[] = "url>;rel=preload;";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // Error if url is not enclosed by <>
+    const char s[] = "<url;rel=preload;";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // Empty parameter value is not allowed
+    const char s[] = "<url>;rel=preload; as=";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // Empty parameter value is not allowed
+    const char s[] = "<url>;as=;rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // Empty parameter value is not allowed
+    const char s[] = "<url>;as=, <url>;rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // Empty parameter name is not allowed
+    const char s[] = "<url>; =file; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // Without whitespaces
+    const char s[] = "<url>;as=file;rel=preload,<url>;rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(2 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+    CU_ASSERT(std::make_pair(&s[27], &s[30]) == res[1].uri);
+  }
+  {
+    // link-extension may have no value
+    const char s[] = "<url>; as; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // ext-name-star
+    const char s[] = "<url>; foo*=bar; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // '*' is not allowed expect for trailing one
+    const char s[] = "<url>; *=bar; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // '*' is not allowed expect for trailing one
+    const char s[] = "<url>; foo*bar=buzz; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // ext-name-star must be followed by '='
+    const char s[] = "<url>; foo*; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // '>' is not followed by ';'
+    const char s[] = "<url> rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // Starting with whitespace is no problem.
+    const char s[] = "  <url>; rel=preload";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[3], &s[6]) == res[0].uri);
+  }
+  {
+    // preload is a prefix of bogus rel parameter value
+    const char s[] = "<url>; rel=preloadx";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // preload in relation-types list
+    const char s[] = R"(<url>; rel="preload")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // preload in relation-types list followed by another parameter
+    const char s[] = R"(<url>; rel="preload foo")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // preload in relation-types list following another parameter
+    const char s[] = R"(<url>; rel="foo preload")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // preload in relation-types list between other parameters
+    const char s[] = R"(<url>; rel="foo preload bar")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // preload in relation-types list between other parameters
+    const char s[] = R"(<url>; rel="foo   preload   bar")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // no preload in relation-types list
+    const char s[] = R"(<url>; rel="foo")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // no preload in relation-types list, multiple unrelated elements.
+    const char s[] = R"(<url>; rel="foo bar")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // preload in relation-types list, followed by another link-value.
+    const char s[] = R"(<url>; rel="preload", <url>)";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // preload in relation-types list, following another link-value.
+    const char s[] = R"(<url>, <url>; rel="preload")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[8], &s[11]) == res[0].uri);
+  }
+  {
+    // preload in relation-types list, followed by another link-param.
+    const char s[] = R"(<url>; rel="preload"; as="font")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // preload in relation-types list, followed by character other
+    // than ';' or ','
+    const char s[] = R"(<url>; rel="preload".)";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // preload in relation-types list, followed by ';' but it
+    // terminates input
+    const char s[] = R"(<url>; rel="preload";)";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // preload in relation-types list, followed by ',' but it
+    // terminates input
+    const char s[] = R"(<url>; rel="preload",)";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // preload in relation-types list but there is preceding white
+    // space.
+    const char s[] = R"(<url>; rel=" preload")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // preload in relation-types list but there is trailing white
+    // space.
+    const char s[] = R"(<url>; rel="preload ")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // backslash escaped characters in quoted-string
+    const char s[] = R"(<url>; rel=preload; title="foo\"baz\"bar")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // anchor="" is acceptable
+    const char s[] = R"(<url>; rel=preload; anchor="")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[1], &s[4]) == res[0].uri);
+  }
+  {
+    // With anchor="#foo", url should be ignored
+    const char s[] = R"(<url>; rel=preload; anchor="#foo")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // With anchor=f, url should be ignored
+    const char s[] = "<url>; rel=preload; anchor=f";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(0 == res.size());
+  }
+  {
+    // First url is ignored With anchor="#foo", but url should be
+    // accepted.
+    const char s[] = R"(<url>; rel=preload; anchor="#foo", <url>; rel=preload)";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(1 == res.size());
+    CU_ASSERT(std::make_pair(&s[36], &s[39]) == res[0].uri);
+  }
+  {
+    // case-insensitive match
+    const char s[] = R"(<url>; rel=preload; ANCHOR="#foo", <url>; )"
+                     R"(REL=PRELOAD, <url>; REL="foo PRELOAD bar")";
+    auto res = http2::parse_link_header(s, sizeof(s) - 1);
+    CU_ASSERT(2 == res.size());
+    CU_ASSERT(std::make_pair(&s[36], &s[39]) == res[0].uri);
+    CU_ASSERT(std::make_pair(&s[42 + 14], &s[42 + 17]) == res[1].uri);
+  }
+}
+
+void test_http2_path_join(void) {
+  {
+    const char base[] = "/";
+    const char rel[] = "/";
+    CU_ASSERT("/" == http2::path_join(base, sizeof(base) - 1, nullptr, 0, rel,
+                                      sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    const char base[] = "/";
+    const char rel[] = "/alpha";
+    CU_ASSERT("/alpha" == http2::path_join(base, sizeof(base) - 1, nullptr, 0,
+                                           rel, sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // rel ends with trailing '/'
+    const char base[] = "/";
+    const char rel[] = "/alpha/";
+    CU_ASSERT("/alpha/" == http2::path_join(base, sizeof(base) - 1, nullptr, 0,
+                                            rel, sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // rel contains multiple components
+    const char base[] = "/";
+    const char rel[] = "/alpha/bravo";
+    CU_ASSERT("/alpha/bravo" == http2::path_join(base, sizeof(base) - 1,
+                                                 nullptr, 0, rel,
+                                                 sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // rel is relative
+    const char base[] = "/";
+    const char rel[] = "alpha/bravo";
+    CU_ASSERT("/alpha/bravo" == http2::path_join(base, sizeof(base) - 1,
+                                                 nullptr, 0, rel,
+                                                 sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // rel is relative and base ends without /, which means it refers
+    // to file.
+    const char base[] = "/alpha";
+    const char rel[] = "bravo/charlie";
+    CU_ASSERT("/bravo/charlie" ==
+              http2::path_join(base, sizeof(base) - 1, nullptr, 0, rel,
+                               sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // rel contains repeated '/'s
+    const char base[] = "/";
+    const char rel[] = "/alpha/////bravo/////";
+    CU_ASSERT("/alpha/bravo/" == http2::path_join(base, sizeof(base) - 1,
+                                                  nullptr, 0, rel,
+                                                  sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // base ends with '/', so '..' eats 'bravo'
+    const char base[] = "/alpha/bravo/";
+    const char rel[] = "../charlie/delta";
+    CU_ASSERT("/alpha/charlie/delta" ==
+              http2::path_join(base, sizeof(base) - 1, nullptr, 0, rel,
+                               sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // base does not end with '/', so '..' eats 'alpha/bravo'
+    const char base[] = "/alpha/bravo";
+    const char rel[] = "../charlie";
+    CU_ASSERT("/charlie" == http2::path_join(base, sizeof(base) - 1, nullptr, 0,
+                                             rel, sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // 'charlie' is eaten by following '..'
+    const char base[] = "/alpha/bravo/";
+    const char rel[] = "../charlie/../delta";
+    CU_ASSERT("/alpha/delta" == http2::path_join(base, sizeof(base) - 1,
+                                                 nullptr, 0, rel,
+                                                 sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // excessive '..' results in '/'
+    const char base[] = "/alpha/bravo/";
+    const char rel[] = "../../../";
+    CU_ASSERT("/" == http2::path_join(base, sizeof(base) - 1, nullptr, 0, rel,
+                                      sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // excessive '..'  and  path component
+    const char base[] = "/alpha/bravo/";
+    const char rel[] = "../../../charlie";
+    CU_ASSERT("/charlie" == http2::path_join(base, sizeof(base) - 1, nullptr, 0,
+                                             rel, sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // rel ends with '..'
+    const char base[] = "/alpha/bravo/";
+    const char rel[] = "charlie/..";
+    CU_ASSERT("/alpha/bravo/" == http2::path_join(base, sizeof(base) - 1,
+                                                  nullptr, 0, rel,
+                                                  sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // base empty and rel contains '..'
+    const char base[] = "";
+    const char rel[] = "charlie/..";
+    CU_ASSERT("/" == http2::path_join(base, sizeof(base) - 1, nullptr, 0, rel,
+                                      sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // '.' is ignored
+    const char base[] = "/";
+    const char rel[] = "charlie/././././delta";
+    CU_ASSERT("/charlie/delta" ==
+              http2::path_join(base, sizeof(base) - 1, nullptr, 0, rel,
+                               sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // trailing '.' is ignored
+    const char base[] = "/";
+    const char rel[] = "charlie/.";
+    CU_ASSERT("/charlie/" == http2::path_join(base, sizeof(base) - 1, nullptr,
+                                              0, rel, sizeof(rel) - 1, nullptr,
+                                              0));
+  }
+  {
+    // query
+    const char base[] = "/";
+    const char rel[] = "/";
+    const char relq[] = "q";
+    CU_ASSERT("/?q" == http2::path_join(base, sizeof(base) - 1, nullptr, 0, rel,
+                                        sizeof(rel) - 1, relq,
+                                        sizeof(relq) - 1));
+  }
+  {
+    // empty rel and query
+    const char base[] = "/alpha";
+    const char rel[] = "";
+    const char relq[] = "q";
+    CU_ASSERT("/alpha?q" == http2::path_join(base, sizeof(base) - 1, nullptr, 0,
+                                             rel, sizeof(rel) - 1, relq,
+                                             sizeof(relq) - 1));
+  }
+  {
+    // both rel and query are empty
+    const char base[] = "/alpha";
+    const char baseq[] = "r";
+    const char rel[] = "";
+    const char relq[] = "";
+    CU_ASSERT("/alpha?r" ==
+              http2::path_join(base, sizeof(base) - 1, baseq, sizeof(baseq) - 1,
+                               rel, sizeof(rel) - 1, relq, sizeof(relq) - 1));
+  }
+  {
+    // empty base
+    const char base[] = "";
+    const char rel[] = "/alpha";
+    CU_ASSERT("/alpha" == http2::path_join(base, sizeof(base) - 1, nullptr, 0,
+                                           rel, sizeof(rel) - 1, nullptr, 0));
+  }
+  {
+    // everything is empty
+    CU_ASSERT("/" ==
+              http2::path_join(nullptr, 0, nullptr, 0, nullptr, 0, nullptr, 0));
+  }
+  {
+    // only baseq is not empty
+    const char base[] = "";
+    const char baseq[] = "r";
+    const char rel[] = "";
+    CU_ASSERT("/?r" == http2::path_join(base, sizeof(base) - 1, baseq,
+                                        sizeof(baseq) - 1, rel, sizeof(rel) - 1,
+                                        nullptr, 0));
   }
 }
 

src/http2_test.h

@@ -39,7 +39,8 @@ void test_http2_lookup_token(void);
 void test_http2_check_http2_pseudo_header(void);
 void test_http2_http2_header_allowed(void);
 void test_http2_mandatory_request_headers_presence(void);
-void test_http2_check_http2_te(void);
+void test_http2_parse_link_header(void);
+void test_http2_path_join(void);
 
 } // namespace shrpx
 

src/memchunk.h

@@ -29,26 +29,32 @@
 
 #include <sys/uio.h>
 
+#include <cassert>
 #include <cstring>
 #include <memory>
+#include <array>
+#include <algorithm>
 
-#include "util.h"
+#include "template.h"
 
 namespace nghttp2 {
 
 template <size_t N> struct Memchunk {
-  Memchunk()
-      : kprev(nullptr), next(nullptr), pos(begin), last(begin), end(begin + N) {
+  Memchunk(std::unique_ptr<Memchunk> next_chunk)
+      : pos(std::begin(buf)), last(pos), knext(std::move(next_chunk)),
+        kprev(nullptr), next(nullptr) {
+    if (knext) {
+      knext->kprev = this;
+    }
   }
   size_t len() const { return last - pos; }
-  size_t left() const { return end - last; }
-  void reset() { pos = last = begin; }
+  size_t left() const { return std::end(buf) - last; }
+  void reset() { pos = last = std::begin(buf); }
+  std::array<uint8_t, N> buf;
+  uint8_t *pos, *last;
   std::unique_ptr<Memchunk> knext;
   Memchunk *kprev;
   Memchunk *next;
-  uint8_t *pos, *last;
-  uint8_t *end;
-  uint8_t begin[N];
   static const size_t size = N;
 };
 
@@ -63,15 +69,9 @@ template <typename T> struct Pool {
       return m;
     }
 
-    auto m = util::make_unique<T>();
-    auto p = m.get();
-    if (pool) {
-      m->knext = std::move(pool);
-      m->knext->kprev = m.get();
-    }
-    pool = std::move(m);
+    pool = make_unique<T>(std::move(pool));
     poolsize += T::size;
-    return p;
+    return pool.get();
   }
   void recycle(T *m) {
     if (freelist) {
@@ -108,11 +108,6 @@ template <typename T> struct Pool {
   size_t poolsize;
 };
 
-inline void *cpymem(void *dest, const void *src, size_t count) {
-  memcpy(dest, src, count);
-  return reinterpret_cast<uint8_t *>(dest) + count;
-}
-
 template <typename Memchunk> struct Memchunks {
   Memchunks(Pool<Memchunk> *pool)
       : pool(pool), head(nullptr), tail(nullptr), len(0) {}
@@ -126,54 +121,53 @@ template <typename Memchunk> struct Memchunks {
       m = next;
     }
   }
-  size_t append(const void *data, size_t count) {
+  size_t append(const void *src, size_t count) {
     if (count == 0) {
       return 0;
     }
 
-    auto p = reinterpret_cast<const uint8_t *>(data);
+    auto first = static_cast<const uint8_t *>(src);
+    auto last = first + count;
 
     if (!tail) {
       head = tail = pool->get();
     }
-    auto all = count;
 
-    while (count > 0) {
-      auto n = std::min(count, tail->left());
-      tail->last = reinterpret_cast<uint8_t *>(cpymem(tail->last, p, n));
-      p += n;
-      count -= n;
+    for (;;) {
+      auto n = std::min(static_cast<size_t>(last - first), tail->left());
+      tail->last = std::copy_n(first, n, tail->last);
+      first += n;
       len += n;
-      if (count == 0) {
+      if (first == last) {
         break;
       }
 
       tail->next = pool->get();
-
-      assert(tail != tail->next);
       tail = tail->next;
     }
 
-    return all;
+    return count;
   }
   template <size_t N> size_t append(const char (&s)[N]) {
     return append(s, N - 1);
   }
-  size_t remove(void *data, size_t count) {
+  size_t remove(void *dest, size_t count) {
     if (!tail || count == 0) {
       return 0;
     }
-    auto ndata = count;
+
+    auto first = static_cast<uint8_t *>(dest);
+    auto last = first + count;
+
     auto m = head;
 
     while (m) {
       auto next = m->next;
-      auto n = std::min(count, m->len());
+      auto n = std::min(static_cast<size_t>(last - first), m->len());
 
       assert(m->len());
-      data = cpymem(data, m->pos, n);
+      first = std::copy_n(m->pos, n, first);
       m->pos += n;
-      count -= n;
       len -= n;
       if (m->len() > 0) {
         break;
@@ -186,7 +180,7 @@ template <typename Memchunk> struct Memchunks {
       tail = nullptr;
     }
 
-    return ndata - count;
+    return first - static_cast<uint8_t *>(dest);
   }
   size_t drain(size_t count) {
     auto ndata = count;
@@ -233,6 +227,29 @@ using Memchunk16K = Memchunk<16384>;
 using MemchunkPool = Pool<Memchunk16K>;
 using DefaultMemchunks = Memchunks<Memchunk16K>;
 
+#define DEFAULT_WR_IOVCNT 16
+
+#if defined(IOV_MAX) && IOV_MAX < DEFAULT_WR_IOVCNT
+#define MAX_WR_IOVCNT IOV_MAX
+#else // !defined(IOV_MAX) || IOV_MAX >= DEFAULT_WR_IOVCNT
+#define MAX_WR_IOVCNT DEFAULT_WR_IOVCNT
+#endif // !defined(IOV_MAX) || IOV_MAX >= DEFAULT_WR_IOVCNT
+
+inline int limit_iovec(struct iovec *iov, int iovcnt, size_t max) {
+  if (max == 0) {
+    return 0;
+  }
+  for (int i = 0; i < iovcnt; ++i) {
+    auto d = std::min(max, iov[i].iov_len);
+    iov[i].iov_len = d;
+    max -= d;
+    if (max == 0) {
+      return i + 1;
+    }
+  }
+  return iovcnt;
+}
+
 } // namespace nghttp2
 
 #endif // MEMCHUNK_H

src/memchunk_test.cc

@@ -29,6 +29,7 @@
 #include <nghttp2/nghttp2.h>
 
 #include "memchunk.h"
+#include "util.h"
 
 namespace nghttp2 {
 
@@ -153,28 +154,28 @@ void test_memchunks_riovec(void) {
 
   chunks.append(buf, sizeof(buf));
 
-  struct iovec iov[2];
-  auto iovcnt = chunks.riovec(iov, util::array_size(iov));
+  std::array<struct iovec, 2> iov;
+  auto iovcnt = chunks.riovec(iov.data(), iov.size());
 
   auto m = chunks.head;
 
   CU_ASSERT(2 == iovcnt);
-  CU_ASSERT(m->begin == iov[0].iov_base);
+  CU_ASSERT(m->buf.data() == iov[0].iov_base);
   CU_ASSERT(m->len() == iov[0].iov_len);
 
   m = m->next;
 
-  CU_ASSERT(m->begin == iov[1].iov_base);
+  CU_ASSERT(m->buf.data() == iov[1].iov_base);
   CU_ASSERT(m->len() == iov[1].iov_len);
 
   chunks.drain(2 * 16);
 
-  iovcnt = chunks.riovec(iov, util::array_size(iov));
+  iovcnt = chunks.riovec(iov.data(), iov.size());
 
   CU_ASSERT(1 == iovcnt);
 
   m = chunks.head;
-  CU_ASSERT(m->begin == iov[0].iov_base);
+  CU_ASSERT(m->buf.data() == iov[0].iov_base);
   CU_ASSERT(m->len() == iov[0].iov_len);
 }
 

src/nghttp.cc

@@ -37,6 +37,7 @@
 #include <cstdlib>
 #include <cstring>
 #include <iostream>
+#include <iomanip>
 #include <sstream>
 #include <tuple>
 
@@ -52,6 +53,7 @@
 #include "util.h"
 #include "base64.h"
 #include "ssl.h"
+#include "template.h"
 
 #ifndef O_BINARY
 #define O_BINARY (0)
@@ -107,20 +109,6 @@ std::string strip_fragment(const char *raw_uri) {
 }
 } // namespace
 
-namespace {
-// Returns numeric address string of |addr|.  If getnameinfo() is
-// failed, "unknown" is returned.
-std::string numeric_name(addrinfo *addr) {
-  char host[NI_MAXHOST];
-  auto rv = getnameinfo(addr->ai_addr, addr->ai_addrlen, host, sizeof(host),
-                        nullptr, 0, NI_NUMERICHOST);
-  if (rv != 0) {
-    return "unknown";
-  }
-  return host;
-}
-} // namespace
-
 Request::Request(const std::string &uri, const http_parser_url &u,
                  const nghttp2_data_provider *data_prd, int64_t data_length,
                  const nghttp2_priority_spec &pri_spec,
@@ -248,7 +236,7 @@ bool Request::is_ipv6_literal_addr() const {
   }
 }
 
-bool Request::response_pseudo_header_allowed(int token) const {
+bool Request::response_pseudo_header_allowed(int16_t token) const {
   if (!res_nva.empty() && res_nva.back().name.c_str()[0] != ':') {
     return false;
   }
@@ -260,7 +248,7 @@ bool Request::response_pseudo_header_allowed(int token) const {
   }
 }
 
-bool Request::push_request_pseudo_header_allowed(int token) const {
+bool Request::push_request_pseudo_header_allowed(int16_t token) const {
   if (!req_nva.empty() && req_nva.back().name.c_str()[0] != ':') {
     return false;
   }
@@ -275,7 +263,7 @@ bool Request::push_request_pseudo_header_allowed(int token) const {
   }
 }
 
-Headers::value_type *Request::get_res_header(int token) {
+Headers::value_type *Request::get_res_header(int16_t token) {
   auto idx = res_hdidx[token];
   if (idx == -1) {
     return nullptr;
@@ -283,7 +271,7 @@ Headers::value_type *Request::get_res_header(int token) {
   return &res_nva[idx];
 }
 
-Headers::value_type *Request::get_req_header(int token) {
+Headers::value_type *Request::get_req_header(int16_t token) {
   auto idx = req_hdidx[token];
   if (idx == -1) {
     return nullptr;
@@ -291,19 +279,19 @@ Headers::value_type *Request::get_req_header(int token) {
   return &req_nva[idx];
 }
 
-void Request::record_request_time() {
-  stat.stage = STAT_ON_REQUEST;
-  stat.on_request_time = get_time();
+void Request::record_request_start_time() {
+  timing.state = RequestState::ON_REQUEST;
+  timing.request_start_time = get_time();
 }
 
-void Request::record_response_time() {
-  stat.stage = STAT_ON_RESPONSE;
-  stat.on_response_time = get_time();
+void Request::record_response_start_time() {
+  timing.state = RequestState::ON_RESPONSE;
+  timing.response_start_time = get_time();
 }
 
-void Request::record_complete_time() {
-  stat.stage = STAT_ON_COMPLETE;
-  stat.on_complete_time = get_time();
+void Request::record_response_end_time() {
+  timing.state = RequestState::ON_COMPLETE;
+  timing.response_end_time = get_time();
 }
 
 namespace {
@@ -452,7 +440,7 @@ HttpClient::HttpClient(const nghttp2_session_callbacks *callbacks,
                        struct ev_loop *loop, SSL_CTX *ssl_ctx)
     : session(nullptr), callbacks(callbacks), loop(loop), ssl_ctx(ssl_ctx),
       ssl(nullptr), addrs(nullptr), next_addr(nullptr), cur_addr(nullptr),
-      complete(0), settings_payloadlen(0), state(STATE_IDLE),
+      complete(0), settings_payloadlen(0), state(ClientState::IDLE),
       upgrade_response_status_code(0), fd(-1),
       upgrade_response_complete(false) {
   ev_io_init(&wev, writecb, 0, EV_WRITE);
@@ -589,7 +577,7 @@ int HttpClient::initiate_connection() {
 }
 
 void HttpClient::disconnect() {
-  state = STATE_IDLE;
+  state = ClientState::IDLE;
 
   ev_timer_stop(loop, &settings_timer);
 
@@ -620,11 +608,11 @@ void HttpClient::disconnect() {
 int HttpClient::read_clear() {
   ev_timer_again(loop, &rt);
 
-  uint8_t buf[8192];
+  std::array<uint8_t, 8192> buf;
 
   for (;;) {
     ssize_t nread;
-    while ((nread = read(fd, buf, sizeof(buf))) == -1 && errno == EINTR)
+    while ((nread = read(fd, buf.data(), buf.size())) == -1 && errno == EINTR)
       ;
     if (nread == -1) {
       if (errno == EAGAIN || errno == EWOULDBLOCK) {
@@ -637,7 +625,7 @@ int HttpClient::read_clear() {
       return -1;
     }
 
-    if (on_readfn(*this, buf, nread) != 0) {
+    if (on_readfn(*this, buf.data(), nread) != 0) {
       return -1;
     }
   }
@@ -650,11 +638,8 @@ int HttpClient::write_clear() {
 
   for (;;) {
     if (wb.rleft() > 0) {
-      struct iovec iov[2];
-      auto iovcnt = wb.riovec(iov);
-
       ssize_t nwrite;
-      while ((nwrite = writev(fd, iov, iovcnt)) == -1 && errno == EINTR)
+      while ((nwrite = write(fd, wb.pos, wb.rleft())) == -1 && errno == EINTR)
         ;
       if (nwrite == -1) {
         if (errno == EAGAIN || errno == EWOULDBLOCK) {
@@ -667,12 +652,11 @@ int HttpClient::write_clear() {
       wb.drain(nwrite);
       continue;
     }
-
+    wb.reset();
     if (on_writefn(*this) != 0) {
       return -1;
     }
     if (wb.rleft() == 0) {
-      wb.reset();
       break;
     }
   }
@@ -686,15 +670,17 @@ int HttpClient::write_clear() {
 int HttpClient::noop() { return 0; }
 
 void HttpClient::on_connect_fail() {
-  if (state == STATE_IDLE) {
+  if (state == ClientState::IDLE) {
     std::cerr << "[ERROR] Could not connect to the address "
-              << numeric_name(cur_addr) << std::endl;
+              << util::numeric_name(cur_addr->ai_addr, cur_addr->ai_addrlen)
+              << std::endl;
   }
   auto cur_state = state;
   disconnect();
-  if (cur_state == STATE_IDLE) {
+  if (cur_state == ClientState::IDLE) {
     if (initiate_connection() == 0) {
-      std::cerr << "Trying next address " << numeric_name(cur_addr)
+      std::cerr << "Trying next address "
+                << util::numeric_name(cur_addr->ai_addr, cur_addr->ai_addrlen)
                 << std::endl;
     }
   }
@@ -710,8 +696,7 @@ int HttpClient::connected() {
     std::cout << " Connected" << std::endl;
   }
 
-  record_connect_time();
-  state = STATE_CONNECTED;
+  state = ClientState::CONNECTED;
 
   ev_io_start(loop, &rev);
   ev_io_stop(loop, &wev);
@@ -730,7 +715,7 @@ int HttpClient::connected() {
   writefn = &HttpClient::write_clear;
 
   if (need_upgrade()) {
-    htp = util::make_unique<http_parser>();
+    htp = make_unique<http_parser>();
     http_parser_init(htp.get(), HTTP_RESPONSE);
     htp->data = this;
 
@@ -769,19 +754,20 @@ size_t populate_settings(nghttp2_settings_entry *iv) {
 
 int HttpClient::on_upgrade_connect() {
   ssize_t rv;
-  record_handshake_time();
+  record_connect_end_time();
   assert(!reqvec.empty());
-  nghttp2_settings_entry iv[32];
-  size_t niv = populate_settings(iv);
-  assert(sizeof(settings_payload) >= 8 * niv);
-  rv = nghttp2_pack_settings_payload(settings_payload, sizeof(settings_payload),
-                                     iv, niv);
+  std::array<nghttp2_settings_entry, 32> iv;
+  size_t niv = populate_settings(iv.data());
+  assert(settings_payload.size() >= 8 * niv);
+  rv = nghttp2_pack_settings_payload(settings_payload.data(),
+                                     settings_payload.size(), iv.data(), niv);
   if (rv < 0) {
     return -1;
   }
   settings_payloadlen = rv;
-  auto token68 = base64::encode(&settings_payload[0],
-                                &settings_payload[settings_payloadlen]);
+  auto token68 =
+      base64::encode(std::begin(settings_payload),
+                     std::begin(settings_payload) + settings_payloadlen);
   util::to_token68(token68);
   std::string req;
   if (reqvec[0]->data_prd) {
@@ -791,18 +777,38 @@ int HttpClient::on_upgrade_connect() {
     req = "GET ";
     req += reqvec[0]->make_reqpath();
   }
-  req += " HTTP/1.1\r\n"
-         "Host: ";
-  req += hostport;
-  req += "\r\n"
-         "Connection: Upgrade, HTTP2-Settings\r\n"
-         "Upgrade: " NGHTTP2_CLEARTEXT_PROTO_VERSION_ID "\r\n"
-         "HTTP2-Settings: ";
-  req += token68;
-  req += "\r\n"
-         "Accept: */*\r\n"
-         "User-Agent: nghttp2/" NGHTTP2_VERSION "\r\n"
-         "\r\n";
+
+  auto headers = Headers{{"Host", hostport},
+                         {"Connection", "Upgrade, HTTP2-Settings"},
+                         {"Upgrade", NGHTTP2_CLEARTEXT_PROTO_VERSION_ID},
+                         {"HTTP2-Settings", token68},
+                         {"Accept", "*/*"},
+                         {"User-Agent", "nghttp2/" NGHTTP2_VERSION}};
+  auto initial_headerslen = headers.size();
+
+  for (auto &kv : config.headers) {
+    size_t i;
+    for (i = 0; i < initial_headerslen; ++i) {
+      if (util::strieq(kv.name, headers[i].name)) {
+        headers[i].value = kv.value;
+        break;
+      }
+    }
+    if (i < initial_headerslen) {
+      continue;
+    }
+    headers.emplace_back(kv.name, kv.value, kv.no_index);
+  }
+
+  req += " HTTP/1.1\r\n";
+
+  for (auto &kv : headers) {
+    req += kv.name;
+    req += ": ";
+    req += kv.value;
+    req += "\r\n";
+  }
+  req += "\r\n";
 
   wb.write(req.c_str(), req.size());
 
@@ -811,6 +817,11 @@ int HttpClient::on_upgrade_connect() {
     std::cout << " HTTP Upgrade request\n" << req << std::endl;
   }
 
+  // record request time if this is GET request
+  if (!reqvec[0]->data_prd) {
+    reqvec[0]->record_request_start_time();
+  }
+
   on_writefn = &HttpClient::noop;
 
   signal_write();
@@ -880,6 +891,10 @@ int HttpClient::do_write() { return writefn(*this); }
 int HttpClient::on_connect() {
   int rv;
 
+  if (!need_upgrade()) {
+    record_connect_end_time();
+  }
+
   if (ssl) {
     // Check NPN or ALPN result
     const unsigned char *next_proto = nullptr;
@@ -910,10 +925,6 @@ int HttpClient::on_connect() {
     }
   }
 
-  if (!need_upgrade()) {
-    record_handshake_time();
-  }
-
   rv = nghttp2_session_client_new2(&session, callbacks, this,
                                    config.http2_option);
 
@@ -927,8 +938,8 @@ int HttpClient::on_connect() {
     if (!reqvec[0]->data_prd) {
       stream_user_data = reqvec[0].get();
     }
-    rv = nghttp2_session_upgrade(session, settings_payload, settings_payloadlen,
-                                 stream_user_data);
+    rv = nghttp2_session_upgrade(session, settings_payload.data(),
+                                 settings_payloadlen, stream_user_data);
     if (rv != 0) {
       std::cerr << "[ERROR] nghttp2_session_upgrade() returned error: "
                 << nghttp2_strerror(rv) << std::endl;
@@ -946,9 +957,9 @@ int HttpClient::on_connect() {
   // HTTP2-Settings header field has already been submitted to
   // session object.
   if (!need_upgrade()) {
-    nghttp2_settings_entry iv[16];
-    auto niv = populate_settings(iv);
-    rv = nghttp2_submit_settings(session, NGHTTP2_FLAG_NONE, iv, niv);
+    std::array<nghttp2_settings_entry, 16> iv;
+    auto niv = populate_settings(iv.data());
+    rv = nghttp2_submit_settings(session, NGHTTP2_FLAG_NONE, iv.data(), niv);
     if (rv != 0) {
       return -1;
     }
@@ -1105,9 +1116,9 @@ int HttpClient::read_tls() {
 
   ERR_clear_error();
 
-  uint8_t buf[8192];
+  std::array<uint8_t, 8192> buf;
   for (;;) {
-    auto rv = SSL_read(ssl, buf, sizeof(buf));
+    auto rv = SSL_read(ssl, buf.data(), buf.size());
 
     if (rv == 0) {
       return -1;
@@ -1126,7 +1137,7 @@ int HttpClient::read_tls() {
       }
     }
 
-    if (on_readfn(*this, buf, rv) != 0) {
+    if (on_readfn(*this, buf.data(), rv) != 0) {
       return -1;
     }
   }
@@ -1139,11 +1150,7 @@ int HttpClient::write_tls() {
 
   for (;;) {
     if (wb.rleft() > 0) {
-      const void *p;
-      size_t len;
-      std::tie(p, len) = wb.get();
-
-      auto rv = SSL_write(ssl, p, len);
+      auto rv = SSL_write(ssl, wb.pos, wb.rleft());
 
       if (rv == 0) {
         return -1;
@@ -1168,6 +1175,7 @@ int HttpClient::write_tls() {
 
       continue;
     }
+    wb.reset();
     if (on_writefn(*this) != 0) {
       return -1;
     }
@@ -1228,29 +1236,25 @@ bool HttpClient::add_request(const std::string &uri,
     path_cache.insert(uri);
   }
 
-  reqvec.push_back(util::make_unique<Request>(
-      uri, u, data_prd, data_length, pri_spec, std::move(dep), pri, level));
+  reqvec.push_back(make_unique<Request>(uri, u, data_prd, data_length, pri_spec,
+                                        std::move(dep), pri, level));
   return true;
 }
 
-void HttpClient::record_handshake_time() {
-  stat.on_handshake_time = get_time();
+void HttpClient::record_start_time() {
+  timing.system_start_time = std::chrono::system_clock::now();
+  timing.start_time = get_time();
 }
 
-void HttpClient::record_started_time() {
-  stat.started_system_time = std::chrono::system_clock::now();
-  stat.on_started_time = get_time();
+void HttpClient::record_domain_lookup_end_time() {
+  timing.domain_lookup_end_time = get_time();
 }
 
-void HttpClient::record_dns_complete_time() {
-  stat.on_dns_complete_time = get_time();
+void HttpClient::record_connect_end_time() {
+  timing.connect_end_time = get_time();
 }
 
-void HttpClient::record_connect_time() { stat.on_connect_time = get_time(); }
-
 void HttpClient::on_request(Request *req) {
-  req->record_request_time();
-
   if (req->pri == 0 && req->dep) {
     assert(req->dep->deps.empty());
 
@@ -1307,7 +1311,7 @@ void HttpClient::output_har(FILE *outfile) {
 
   json_object_set_new(
       page, "startedDateTime",
-      json_string(util::format_iso8601(stat.started_system_time).c_str()));
+      json_string(util::format_iso8601(timing.system_start_time).c_str()));
   json_object_set_new(page, "id", json_string(PAGE_ID));
   json_object_set_new(page, "title", json_string(""));
 
@@ -1318,45 +1322,45 @@ void HttpClient::output_har(FILE *outfile) {
 
   auto dns_delta =
       std::chrono::duration_cast<std::chrono::microseconds>(
-          stat.on_dns_complete_time - stat.on_started_time).count() /
+          timing.domain_lookup_end_time - timing.start_time).count() /
       1000.0;
   auto connect_delta =
       std::chrono::duration_cast<std::chrono::microseconds>(
-          stat.on_connect_time - stat.on_dns_complete_time).count() /
+          timing.connect_end_time - timing.domain_lookup_end_time).count() /
       1000.0;
 
   for (size_t i = 0; i < reqvec.size(); ++i) {
     auto &req = reqvec[i];
 
-    if (req->stat.stage != STAT_ON_COMPLETE) {
+    if (req->timing.state != RequestState::ON_COMPLETE) {
       continue;
     }
 
     auto entry = json_object();
     json_array_append_new(entries, entry);
 
-    auto &req_stat = req->stat;
+    auto &req_timing = req->timing;
     auto request_time =
-        (i == 0) ? stat.started_system_time
-                 : stat.started_system_time +
+        (i == 0) ? timing.system_start_time
+                 : timing.system_start_time +
                        std::chrono::duration_cast<
                            std::chrono::system_clock::duration>(
-                           req_stat.on_request_time - stat.on_started_time);
+                           req_timing.request_start_time - timing.start_time);
 
-    auto wait_delta =
-        std::chrono::duration_cast<std::chrono::microseconds>(
-            req_stat.on_response_time - req_stat.on_request_time).count() /
-        1000.0;
-    auto receive_delta =
-        std::chrono::duration_cast<std::chrono::microseconds>(
-            req_stat.on_complete_time - req_stat.on_response_time).count() /
-        1000.0;
+    auto wait_delta = std::chrono::duration_cast<std::chrono::microseconds>(
+                          req_timing.response_start_time -
+                          req_timing.request_start_time).count() /
+                      1000.0;
+    auto receive_delta = std::chrono::duration_cast<std::chrono::microseconds>(
+                             req_timing.response_end_time -
+                             req_timing.response_start_time).count() /
+                         1000.0;
 
     auto time_sum =
         std::chrono::duration_cast<std::chrono::microseconds>(
-            (i == 0) ? (req_stat.on_complete_time - stat.on_started_time)
-                     : (req_stat.on_complete_time - req_stat.on_request_time))
-            .count() /
+            (i == 0) ? (req_timing.response_end_time - timing.start_time)
+                     : (req_timing.response_end_time -
+                        req_timing.request_start_time)).count() /
         1000.0;
 
     json_object_set_new(
@@ -1509,32 +1513,27 @@ int on_data_chunk_recv_callback(nghttp2_session *session, uint8_t flags,
                                         user_data);
   }
 
-  if (req->status == 0) {
-    nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE, stream_id,
-                              NGHTTP2_PROTOCOL_ERROR);
-    return 0;
-  }
+  req->response_len += len;
 
   if (req->inflater) {
     while (len > 0) {
       const size_t MAX_OUTLEN = 4096;
-      uint8_t out[MAX_OUTLEN];
+      std::array<uint8_t, MAX_OUTLEN> out;
       size_t outlen = MAX_OUTLEN;
       size_t tlen = len;
-      int rv = nghttp2_gzip_inflate(req->inflater, out, &outlen, data, &tlen);
+      int rv =
+          nghttp2_gzip_inflate(req->inflater, out.data(), &outlen, data, &tlen);
       if (rv != 0) {
         nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE, stream_id,
                                   NGHTTP2_INTERNAL_ERROR);
         break;
       }
 
-      req->response_len += outlen;
-
       if (!config.null_out) {
-        std::cout.write(reinterpret_cast<const char *>(out), outlen);
+        std::cout.write(reinterpret_cast<const char *>(out.data()), outlen);
       }
 
-      update_html_parser(client, req, out, outlen, 0);
+      update_html_parser(client, req, out.data(), outlen, 0);
       data += tlen;
       len -= tlen;
     }
@@ -1542,8 +1541,6 @@ int on_data_chunk_recv_callback(nghttp2_session *session, uint8_t flags,
     return 0;
   }
 
-  req->response_len += len;
-
   if (!config.null_out) {
     std::cout.write(reinterpret_cast<const char *>(data), len);
   }
@@ -1587,8 +1584,8 @@ void check_response_header(nghttp2_session *session, Request *req) {
 
   for (auto &nv : req->res_nva) {
     if ("content-encoding" == nv.name) {
-      gzip =
-          util::strieq("gzip", nv.value) || util::strieq("deflate", nv.value);
+      gzip = util::strieq_l("gzip", nv.value) ||
+             util::strieq_l("deflate", nv.value);
       continue;
     }
   }
@@ -1619,6 +1616,24 @@ int on_begin_headers_callback(nghttp2_session *session,
                               const nghttp2_frame *frame, void *user_data) {
   auto client = get_client(user_data);
   switch (frame->hd.type) {
+  case NGHTTP2_HEADERS: {
+    auto req = static_cast<Request *>(
+        nghttp2_session_get_stream_user_data(session, frame->hd.stream_id));
+    if (!req) {
+      break;
+    }
+
+    switch (frame->headers.cat) {
+    case NGHTTP2_HCAT_RESPONSE:
+    case NGHTTP2_HCAT_PUSH_RESPONSE:
+      req->record_response_start_time();
+      break;
+    default:
+      break;
+    }
+
+    break;
+  }
   case NGHTTP2_PUSH_PROMISE: {
     auto stream_id = frame->push_promise.promised_stream_id;
     http_parser_url u;
@@ -1628,12 +1643,13 @@ int on_begin_headers_callback(nghttp2_session *session,
 
     nghttp2_priority_spec_default_init(&pri_spec);
 
-    auto req = util::make_unique<Request>("", u, nullptr, 0, pri_spec, nullptr);
+    auto req = make_unique<Request>("", u, nullptr, 0, pri_spec, nullptr);
     req->stream_id = stream_id;
 
     nghttp2_session_set_stream_user_data(session, stream_id, req.get());
 
     client->on_request(req.get());
+    req->record_request_start_time();
     client->reqvec.push_back(std::move(req));
 
     break;
@@ -1653,12 +1669,6 @@ int on_header_callback(nghttp2_session *session, const nghttp2_frame *frame,
                                flags, user_data);
   }
 
-  if (!http2::check_nv(name, namelen, value, valuelen)) {
-    nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE, frame->hd.stream_id,
-                              NGHTTP2_PROTOCOL_ERROR);
-    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-  }
-
   switch (frame->hd.type) {
   case NGHTTP2_HEADERS: {
     auto req = static_cast<Request *>(
@@ -1668,26 +1678,17 @@ int on_header_callback(nghttp2_session *session, const nghttp2_frame *frame,
       break;
     }
 
-    if (frame->headers.cat != NGHTTP2_HCAT_RESPONSE &&
-        frame->headers.cat != NGHTTP2_HCAT_PUSH_RESPONSE &&
-        (frame->headers.cat != NGHTTP2_HCAT_HEADERS ||
-         !req->expect_final_response)) {
+    /* ignore trailer header */
+    if (frame->headers.cat == NGHTTP2_HCAT_HEADERS &&
+        !req->expect_final_response) {
       break;
     }
 
     auto token = http2::lookup_token(name, namelen);
 
-    if (name[0] == ':') {
-      if (!req->response_pseudo_header_allowed(token)) {
-        nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE,
-                                  frame->hd.stream_id, NGHTTP2_PROTOCOL_ERROR);
-        return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
-      }
-    }
-
     http2::index_header(req->res_hdidx, token, req->res_nva.size());
     http2::add_header(req->res_nva, name, namelen, value, valuelen,
-                      flags & NGHTTP2_NV_FLAG_NO_INDEX);
+                      flags & NGHTTP2_NV_FLAG_NO_INDEX, token);
     break;
   }
   case NGHTTP2_PUSH_PROMISE: {
@@ -1700,18 +1701,9 @@ int on_header_callback(nghttp2_session *session, const nghttp2_frame *frame,
 
     auto token = http2::lookup_token(name, namelen);
 
-    if (name[0] == ':') {
-      if (!req->push_request_pseudo_header_allowed(token)) {
-        nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE,
-                                  frame->push_promise.promised_stream_id,
-                                  NGHTTP2_PROTOCOL_ERROR);
-        break;
-      }
-    }
-
     http2::index_header(req->req_hdidx, token, req->req_nva.size());
     http2::add_header(req->req_nva, name, namelen, value, valuelen,
-                      flags & NGHTTP2_NV_FLAG_NO_INDEX);
+                      flags & NGHTTP2_NV_FLAG_NO_INDEX, token);
     break;
   }
   }
@@ -1730,37 +1722,52 @@ int on_frame_recv_callback2(nghttp2_session *session,
 
   auto client = get_client(user_data);
   switch (frame->hd.type) {
+  case NGHTTP2_DATA: {
+    auto req = static_cast<Request *>(
+        nghttp2_session_get_stream_user_data(session, frame->hd.stream_id));
+    if (!req) {
+      return 0;
+      ;
+    }
+
+    if (frame->hd.flags & NGHTTP2_FLAG_END_STREAM) {
+      req->record_response_end_time();
+    }
+
+    break;
+  }
   case NGHTTP2_HEADERS: {
     auto req = static_cast<Request *>(
         nghttp2_session_get_stream_user_data(session, frame->hd.stream_id));
     // If this is the HTTP Upgrade with OPTIONS method to avoid POST,
     // req is nullptr.
     if (!req) {
-      break;
+      return 0;
+      ;
     }
 
-    if (frame->headers.cat == NGHTTP2_HCAT_RESPONSE ||
-        frame->headers.cat == NGHTTP2_HCAT_PUSH_RESPONSE) {
-      req->record_response_time();
+    switch (frame->headers.cat) {
+    case NGHTTP2_HCAT_RESPONSE:
+    case NGHTTP2_HCAT_PUSH_RESPONSE:
       check_response_header(session, req);
-
       break;
-    }
-
-    if (frame->headers.cat == NGHTTP2_HCAT_HEADERS) {
+    case NGHTTP2_HCAT_HEADERS:
       if (req->expect_final_response) {
         check_response_header(session, req);
-      } else {
-        nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE,
-                                  frame->hd.stream_id, NGHTTP2_PROTOCOL_ERROR);
         break;
       }
+      if ((frame->hd.flags & NGHTTP2_FLAG_END_STREAM) == 0) {
+        nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE,
+                                  frame->hd.stream_id, NGHTTP2_PROTOCOL_ERROR);
+        return 0;
+      }
+      break;
+    default:
+      assert(0);
     }
 
-    if (req->status == 0 && (frame->hd.flags & NGHTTP2_FLAG_END_STREAM)) {
-      nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE, frame->hd.stream_id,
-                                NGHTTP2_PROTOCOL_ERROR);
-      break;
+    if (frame->hd.flags & NGHTTP2_FLAG_END_STREAM) {
+      req->record_response_end_time();
     }
 
     break;
@@ -1779,16 +1786,15 @@ int on_frame_recv_callback2(nghttp2_session *session,
     }
     auto scheme = req->get_req_header(http2::HD__SCHEME);
     auto authority = req->get_req_header(http2::HD__AUTHORITY);
-    auto method = req->get_req_header(http2::HD__METHOD);
     auto path = req->get_req_header(http2::HD__PATH);
 
     if (!authority) {
       authority = req->get_req_header(http2::HD_HOST);
     }
 
-    if (!scheme || !authority || !method || !path || scheme->value.empty() ||
-        authority->value.empty() || method->value.empty() ||
-        path->value.empty() || path->value[0] != '/') {
+    // libnghttp2 guarantees :scheme, :method, :path and (:authority |
+    // host) exist and non-empty.
+    if (path->value[0] != '/') {
       nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE,
                                 frame->push_promise.promised_stream_id,
                                 NGHTTP2_PROTOCOL_ERROR);
@@ -1808,6 +1814,18 @@ int on_frame_recv_callback2(nghttp2_session *session,
     }
     req->uri = uri;
     req->u = u;
+
+    if (client->path_cache.count(uri)) {
+      nghttp2_submit_rst_stream(session, NGHTTP2_FLAG_NONE,
+                                frame->push_promise.promised_stream_id,
+                                NGHTTP2_CANCEL);
+      break;
+    }
+
+    if (config.multiply == 1) {
+      client->path_cache.insert(uri);
+    }
+
     break;
   }
   }
@@ -1815,6 +1833,22 @@ int on_frame_recv_callback2(nghttp2_session *session,
 }
 } // namespace
 
+namespace {
+int before_frame_send_callback(nghttp2_session *session,
+                               const nghttp2_frame *frame, void *user_data) {
+  if (frame->hd.type != NGHTTP2_HEADERS ||
+      frame->headers.cat != NGHTTP2_HCAT_REQUEST) {
+    return 0;
+  }
+  auto req = static_cast<Request *>(
+      nghttp2_session_get_stream_user_data(session, frame->hd.stream_id));
+  assert(req);
+  req->record_request_start_time();
+  return 0;
+}
+
+} // namespace
+
 namespace {
 int on_stream_close_callback(nghttp2_session *session, int32_t stream_id,
                              uint32_t error_code, void *user_data) {
@@ -1827,7 +1861,6 @@ int on_stream_close_callback(nghttp2_session *session, int32_t stream_id,
   }
 
   update_html_parser(client, req, nullptr, 0, 1);
-  req->record_complete_time();
   ++client->complete;
 
   if (client->all_requests_processed()) {
@@ -1838,31 +1871,67 @@ int on_stream_close_callback(nghttp2_session *session, int32_t stream_id,
 }
 } // namespace
 
+struct RequestResult {
+  std::chrono::microseconds time;
+};
+
 namespace {
 void print_stats(const HttpClient &client) {
   std::cout << "***** Statistics *****" << std::endl;
-  int i = 0;
-  for (auto &req : client.reqvec) {
-    std::cout << "#" << ++i << ": " << req->uri << std::endl;
-    std::cout << "    Status: " << req->status << std::endl;
-    std::cout << "    Delta (ms) from handshake(HEADERS):" << std::endl;
-    if (req->stat.stage >= STAT_ON_RESPONSE) {
-      std::cout << "        response HEADERS: "
-                << time_delta(req->stat.on_response_time,
-                              client.stat.on_handshake_time).count() << "("
-                << time_delta(req->stat.on_response_time,
-                              req->stat.on_request_time).count() << ")"
-                << std::endl;
+
+  std::vector<Request *> reqs;
+  reqs.reserve(client.reqvec.size());
+  for (const auto &req : client.reqvec) {
+    if (req->timing.state == RequestState::ON_COMPLETE) {
+      reqs.push_back(req.get());
     }
-    if (req->stat.stage >= STAT_ON_COMPLETE) {
-      std::cout << "        Completed: "
-                << time_delta(req->stat.on_complete_time,
-                              client.stat.on_handshake_time).count() << "("
-                << time_delta(req->stat.on_complete_time,
-                              req->stat.on_request_time).count() << ")"
-                << std::endl;
-    }
-    std::cout << std::endl;
+  }
+
+  std::sort(std::begin(reqs), std::end(reqs),
+            [](const Request *lhs, const Request *rhs) {
+    const auto &ltiming = lhs->timing;
+    const auto &rtiming = rhs->timing;
+    return ltiming.response_end_time < rtiming.response_end_time ||
+           (ltiming.response_end_time == rtiming.response_end_time &&
+            ltiming.request_start_time < rtiming.request_start_time);
+  });
+
+  std::cout << R"(
+Request timing:
+  responseEnd: the  time  when  last  byte of  response  was  received
+               relative to connectEnd
+ requestStart: the time  just before  first byte  of request  was sent
+               relative  to connectEnd.   If  '*' is  shown, this  was
+               pushed by server.
+      process: responseEnd - requestStart
+         code: HTTP status code
+         size: number  of  bytes  received as  response  body  without
+               inflation.
+          URI: request URI
+
+see http://www.w3.org/TR/resource-timing/#processing-model
+
+sorted by 'complete'
+
+responseEnd requestStart  process code size request path)" << std::endl;
+
+  const auto &base = client.timing.connect_end_time;
+  for (const auto &req : reqs) {
+    auto response_end = std::chrono::duration_cast<std::chrono::microseconds>(
+        req->timing.response_end_time - base);
+    auto request_start = std::chrono::duration_cast<std::chrono::microseconds>(
+        req->timing.request_start_time - base);
+    auto total = std::chrono::duration_cast<std::chrono::microseconds>(
+        req->timing.response_end_time - req->timing.request_start_time);
+    auto pushed = req->stream_id % 2 == 0;
+
+    std::cout << std::setw(11) << ("+" + util::format_duration(response_end))
+              << " " << (pushed ? "*" : " ") << std::setw(11)
+              << ("+" + util::format_duration(request_start)) << " "
+              << std::setw(8) << util::format_duration(total) << " "
+              << std::setw(4) << req->status << " " << std::setw(4)
+              << util::utos_with_unit(req->response_len) << " "
+              << req->make_reqpath() << std::endl;
   }
 }
 } // namespace
@@ -1985,13 +2054,13 @@ int communicate(
     }
     client.update_hostport();
 
-    client.record_started_time();
+    client.record_start_time();
 
     if (client.resolve_host(host, port) != 0) {
       goto fin;
     }
 
-    client.record_dns_complete_time();
+    client.record_domain_lookup_end_time();
 
     if (client.initiate_connection() != 0) {
       goto fin;
@@ -2084,7 +2153,7 @@ int run(char **uris, int n) {
   nghttp2_session_callbacks *callbacks;
 
   nghttp2_session_callbacks_new(&callbacks);
-  auto cbsdel = util::defer(callbacks, nghttp2_session_callbacks_del);
+  auto cbsdel = defer(nghttp2_session_callbacks_del, callbacks);
 
   nghttp2_session_callbacks_set_on_stream_close_callback(
       callbacks, on_stream_close_callback);
@@ -2109,6 +2178,9 @@ int run(char **uris, int n) {
   nghttp2_session_callbacks_set_on_header_callback(callbacks,
                                                    on_header_callback);
 
+  nghttp2_session_callbacks_set_before_frame_send_callback(
+      callbacks, before_frame_send_callback);
+
   nghttp2_session_callbacks_set_send_callback(callbacks, send_callback);
 
   if (config.padding) {
@@ -2144,9 +2216,10 @@ int run(char **uris, int n) {
                     << std::endl;
         }
         while (1) {
-          char buf[1024];
+          std::array<char, 1024> buf;
           ssize_t rret, wret;
-          while ((rret = read(0, buf, sizeof(buf))) == -1 && errno == EINTR)
+          while ((rret = read(0, buf.data(), buf.size())) == -1 &&
+                 errno == EINTR)
             ;
           if (rret == 0)
             break;
@@ -2155,7 +2228,8 @@ int run(char **uris, int n) {
                       << std::endl;
             return 1;
           }
-          while ((wret = write(data_fd, buf, rret)) == -1 && errno == EINTR)
+          while ((wret = write(data_fd, buf.data(), rret)) == -1 &&
+                 errno == EINTR)
             ;
           if (wret != rret) {
             std::cerr << "[ERROR] I/O error while writing to temporary file"
@@ -2446,7 +2520,7 @@ int main(int argc, char **argv) {
       }
       // To test "never index" repr, don't index authorization header
       // field unconditionally.
-      auto no_index = util::strieq("authorization", header);
+      auto no_index = util::strieq_l("authorization", header);
       config.headers.emplace_back(header, value, no_index);
       util::inp_strlower(config.headers.back().name);
       break;

src/nghttp.h

@@ -45,7 +45,7 @@
 
 #include "http-parser/http_parser.h"
 
-#include "ringbuf.h"
+#include "buffer.h"
 #include "http2.h"
 #include "nghttp2_gzip.h"
 
@@ -85,19 +85,20 @@ struct Config {
   bool dep_idle;
 };
 
-enum StatStage {
-  STAT_INITIAL,
-  STAT_ON_REQUEST,
-  STAT_ON_RESPONSE,
-  STAT_ON_COMPLETE
-};
+enum class RequestState { INITIAL, ON_REQUEST, ON_RESPONSE, ON_COMPLETE };
 
-struct RequestStat {
-  std::chrono::steady_clock::time_point on_request_time;
-  std::chrono::steady_clock::time_point on_response_time;
-  std::chrono::steady_clock::time_point on_complete_time;
-  StatStage stage;
-  RequestStat() : stage(STAT_INITIAL) {}
+struct RequestTiming {
+  // The point in time when request is started to be sent.
+  // Corresponds to requestStart in Resource Timing TR.
+  std::chrono::steady_clock::time_point request_start_time;
+  // The point in time when first byte of response is received.
+  // Corresponds to responseStart in Resource Timing TR.
+  std::chrono::steady_clock::time_point response_start_time;
+  // The point in time when last byte of response is received.
+  // Corresponds to responseEnd in Resource Timing TR.
+  std::chrono::steady_clock::time_point response_end_time;
+  RequestState state;
+  RequestTiming() : state(RequestState::INITIAL) {}
 };
 
 struct Request;
@@ -127,15 +128,15 @@ struct Request {
 
   bool is_ipv6_literal_addr() const;
 
-  bool response_pseudo_header_allowed(int token) const;
-  bool push_request_pseudo_header_allowed(int token) const;
+  bool response_pseudo_header_allowed(int16_t token) const;
+  bool push_request_pseudo_header_allowed(int16_t token) const;
 
-  Headers::value_type *get_res_header(int token);
-  Headers::value_type *get_req_header(int token);
+  Headers::value_type *get_res_header(int16_t token);
+  Headers::value_type *get_req_header(int16_t token);
 
-  void record_request_time();
-  void record_response_time();
-  void record_complete_time();
+  void record_request_start_time();
+  void record_response_start_time();
+  void record_response_end_time();
 
   Headers res_nva;
   Headers req_nva;
@@ -144,7 +145,7 @@ struct Request {
   http_parser_url u;
   std::shared_ptr<Dependency> dep;
   nghttp2_priority_spec pri_spec;
-  RequestStat stat;
+  RequestTiming timing;
   int64_t data_length;
   int64_t data_offset;
   // Number of bytes received from server
@@ -158,27 +159,28 @@ struct Request {
   int level;
   // RequestPriority value defined in HtmlParser.h
   int pri;
-  int res_hdidx[http2::HD_MAXIDX];
+  http2::HeaderIndex res_hdidx;
   // used for incoming PUSH_PROMISE
-  int req_hdidx[http2::HD_MAXIDX];
+  http2::HeaderIndex req_hdidx;
   bool expect_final_response;
 };
 
-struct SessionStat {
-  // The point in time when download was started.
-  std::chrono::system_clock::time_point started_system_time;
-  // The point of time when download was started.
-  std::chrono::steady_clock::time_point on_started_time;
-  // The point of time when DNS resolution was completed.
-  std::chrono::steady_clock::time_point on_dns_complete_time;
-  // The point of time when connection was established or SSL/TLS
-  // handshake was completed.
-  std::chrono::steady_clock::time_point on_connect_time;
-  // The point of time when HTTP/2 commnucation was started.
-  std::chrono::steady_clock::time_point on_handshake_time;
+struct SessionTiming {
+  // The point in time when operation was started.  Corresponds to
+  // startTime in Resource Timing TR, but recorded in system clock time.
+  std::chrono::system_clock::time_point system_start_time;
+  // Same as above, but recorded in steady clock time.
+  std::chrono::steady_clock::time_point start_time;
+  // The point in time when DNS resolution was completed.  Corresponds
+  // to domainLookupEnd in Resource Timing TR.
+  std::chrono::steady_clock::time_point domain_lookup_end_time;
+  // The point in time when connection was established or SSL/TLS
+  // handshake was completed.  Corresponds to connectEnd in Resource
+  // Timing TR.
+  std::chrono::steady_clock::time_point connect_end_time;
 };
 
-enum client_state { STATE_IDLE, STATE_CONNECTED };
+enum class ClientState { IDLE, CONNECTED };
 
 struct HttpClient {
   HttpClient(const nghttp2_session_callbacks *callbacks, struct ev_loop *loop,
@@ -220,10 +222,9 @@ struct HttpClient {
                    const nghttp2_priority_spec &pri_spec,
                    std::shared_ptr<Dependency> dep, int pri = 0, int level = 0);
 
-  void record_handshake_time();
-  void record_started_time();
-  void record_dns_complete_time();
-  void record_connect_time();
+  void record_start_time();
+  void record_domain_lookup_end_time();
+  void record_connect_end_time();
 
 #ifdef HAVE_JANSSON
   void output_har(FILE *outfile);
@@ -238,7 +239,7 @@ struct HttpClient {
   std::string hostport;
   // Used for parse the HTTP upgrade response from server
   std::unique_ptr<http_parser> htp;
-  SessionStat stat;
+  SessionTiming timing;
   ev_io wev;
   ev_io rev;
   ev_timer wt;
@@ -259,16 +260,16 @@ struct HttpClient {
   size_t complete;
   // The length of settings_payload
   size_t settings_payloadlen;
-  client_state state;
+  ClientState state;
   // The HTTP status code of the response message of HTTP Upgrade.
   unsigned int upgrade_response_status_code;
   int fd;
   // true if the response message of HTTP Upgrade request is fully
   // received. It is not relevant the upgrade succeeds, or not.
   bool upgrade_response_complete;
-  RingBuf<65536> wb;
+  Buffer<65536> wb;
   // SETTINGS payload sent as token68 in HTTP Upgrade
-  uint8_t settings_payload[128];
+  std::array<uint8_t, 128> settings_payload;
 
   enum { ERR_CONNECT_FAIL = -100 };
 };

src/nghttpd.cc

@@ -53,9 +53,9 @@ int parse_push_config(Config &config, const char *optarg) {
   if (eq == NULL) {
     return -1;
   }
-  auto paths = std::vector<std::string>();
+  auto &paths = config.push[std::string(optarg, eq)];
   auto optarg_end = optarg + strlen(optarg);
-  const char *i = eq + 1;
+  auto i = eq + 1;
   for (;;) {
     const char *j = strchr(i, ',');
     if (j == NULL) {
@@ -68,7 +68,7 @@ int parse_push_config(Config &config, const char *optarg) {
     i = j;
     ++i;
   }
-  config.push[std::string(optarg, eq)] = std::move(paths);
+
   return 0;
 }
 } // namespace
@@ -97,6 +97,9 @@ void print_help(std::ostream &out) {
   <CERT>      Set  path  to  server's  certificate.   Required  unless
               --no-tls is specified.
 Options:
+  -a, --address=<ADDR>
+              The address to bind to.  If not specified the default IP
+              address determined by getaddrinfo is used.
   -D, --daemon
               Run in a background.  If -D is used, the current working
               directory is  changed to '/'.  Therefore  if this option
@@ -151,6 +154,7 @@ int main(int argc, char **argv) {
   while (1) {
     static int flag = 0;
     static option long_options[] = {
+        {"address", required_argument, nullptr, 'a'},
         {"daemon", no_argument, nullptr, 'D'},
         {"htdocs", required_argument, nullptr, 'd'},
         {"help", no_argument, nullptr, 'h'},
@@ -168,13 +172,16 @@ int main(int argc, char **argv) {
         {"early-response", no_argument, &flag, 5},
         {nullptr, 0, nullptr, 0}};
     int option_index = 0;
-    int c =
-        getopt_long(argc, argv, "DVb:c:d:ehn:p:v", long_options, &option_index);
+    int c = getopt_long(argc, argv, "DVb:c:d:ehn:p:va:", long_options,
+                        &option_index);
     char *end;
     if (c == -1) {
       break;
     }
     switch (c) {
+    case 'a':
+      config.address = optarg;
+      break;
     case 'D':
       config.daemon = true;
       break;
@@ -298,7 +305,9 @@ int main(int argc, char **argv) {
   reset_timer();
 
   HttpServer server(&config);
-  server.run();
+  if (server.run() != 0) {
+    exit(EXIT_FAILURE);
+  }
   return 0;
 }
 

src/ringbuf.h

@@ -1,143 +0,0 @@
-/*
- * nghttp2 - HTTP/2 C Library
- *
- * Copyright (c) 2014 Tatsuhiro Tsujikawa
- *
- * Permission is hereby granted, free of charge, to any person obtaining
- * a copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sublicense, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice shall be
- * included in all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
- * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
- * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
- * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- */
-#ifndef RINGBUF_H
-#define RINGBUF_H
-
-#include "nghttp2_config.h"
-
-#include <sys/uio.h>
-
-#include <cstring>
-#include <algorithm>
-
-namespace nghttp2 {
-
-template <size_t N> struct RingBuf {
-  RingBuf() : pos(0), len(0) {}
-  // Returns the number of bytes to read.
-  size_t rleft() const { return len; }
-  // Returns the number of bytes this buffer can store.
-  size_t wleft() const { return N - len; }
-  // Writes up to min(wleft(), |count|) bytes from buffer pointed by
-  // |buf|.  Returns number of bytes written.
-  size_t write(const void *buf, size_t count) {
-    count = std::min(count, wleft());
-    auto last = (pos + len) % N;
-    if (count > N - last) {
-      auto c = N - last;
-      memcpy(begin + last, buf, c);
-      memcpy(begin, reinterpret_cast<const uint8_t *>(buf) + c, count - c);
-    } else {
-      memcpy(begin + last, buf, count);
-    }
-    len += count;
-    return count;
-  }
-  size_t write(size_t count) {
-    count = std::min(count, wleft());
-    len += count;
-    return count;
-  }
-  // Drains min(rleft(), |count|) bytes from start of the buffer.
-  size_t drain(size_t count) {
-    count = std::min(count, rleft());
-    pos = (pos + count) % N;
-    len -= count;
-    return count;
-  }
-  // Returns pointer to the next contiguous readable buffer and its
-  // length.
-  std::pair<const void *, size_t> get() const {
-    if (pos + len > N) {
-      return {begin + pos, N - pos};
-    }
-    return {begin + pos, len};
-  }
-  void reset() { pos = len = 0; }
-  // Fills |iov| for reading.  |iov| must contain at least 2 elements.
-  // Returns the number of filled elements.
-  int riovec(struct iovec *iov) {
-    if (len == 0) {
-      return 0;
-    }
-    if (pos + len > N) {
-      auto c = N - pos;
-      iov[0].iov_base = begin + pos;
-      iov[0].iov_len = c;
-      iov[1].iov_base = begin;
-      iov[1].iov_len = len - c;
-      return 2;
-    }
-    iov[0].iov_base = begin + pos;
-    iov[0].iov_len = len;
-    return 1;
-  }
-  // Fills |iov| for writing.  |iov| must contain at least 2 elements.
-  // Returns the number of filled elements.
-  int wiovec(struct iovec *iov) {
-    if (len == N) {
-      return 0;
-    }
-    if (pos == 0) {
-      iov[0].iov_base = begin + pos + len;
-      iov[0].iov_len = N - pos - len;
-      return 1;
-    }
-    if (pos + len < N) {
-      auto c = N - pos - len;
-      iov[0].iov_base = begin + pos + len;
-      iov[0].iov_len = c;
-      iov[1].iov_base = begin;
-      iov[1].iov_len = N - len - c;
-      return 2;
-    }
-    auto last = (pos + len) % N;
-    iov[0].iov_base = begin + last;
-    iov[0].iov_len = N - len;
-    return 1;
-  }
-  size_t pos;
-  size_t len;
-  uint8_t begin[N];
-};
-
-inline int limit_iovec(struct iovec *iov, int iovcnt, size_t max) {
-  if (max == 0) {
-    return 0;
-  }
-  for (int i = 0; i < iovcnt; ++i) {
-    auto d = std::min(max, iov[i].iov_len);
-    iov[i].iov_len = d;
-    max -= d;
-    if (max == 0) {
-      return i + 1;
-    }
-  }
-  return iovcnt;
-}
-
-} // namespace nghttp2
-
-#endif // RINGBUF_H

src/ringbuf_test.cc

@@ -1,183 +0,0 @@
-/*
- * nghttp2 - HTTP/2 C Library
- *
- * Copyright (c) 2014 Tatsuhiro Tsujikawa
- *
- * Permission is hereby granted, free of charge, to any person obtaining
- * a copy of this software and associated documentation files (the
- * "Software"), to deal in the Software without restriction, including
- * without limitation the rights to use, copy, modify, merge, publish,
- * distribute, sublicense, and/or sell copies of the Software, and to
- * permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice shall be
- * included in all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
- * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
- * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
- * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
- * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
- * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- */
-#include "ringbuf_test.h"
-
-#include <cstring>
-#include <iostream>
-#include <tuple>
-
-#include <CUnit/CUnit.h>
-
-#include <nghttp2/nghttp2.h>
-
-#include "ringbuf.h"
-
-namespace nghttp2 {
-
-void test_ringbuf_write(void) {
-  RingBuf<16> b;
-  CU_ASSERT(0 == b.rleft());
-  CU_ASSERT(16 == b.wleft());
-
-  b.write("012", 3);
-
-  CU_ASSERT(3 == b.rleft());
-  CU_ASSERT(13 == b.wleft());
-  CU_ASSERT(0 == b.pos);
-  CU_ASSERT(3 == b.len);
-
-  b.drain(3);
-
-  CU_ASSERT(0 == b.rleft());
-  CU_ASSERT(16 == b.wleft());
-  CU_ASSERT(3 == b.pos);
-  CU_ASSERT(0 == b.len);
-
-  b.write("0123456789ABCDEF", 16);
-
-  CU_ASSERT(16 == b.rleft());
-  CU_ASSERT(0 == b.wleft());
-  CU_ASSERT(3 == b.pos);
-  CU_ASSERT(16 == b.len);
-  CU_ASSERT(0 == memcmp(b.begin, "DEF0123456789ABC", 16));
-
-  const void *p;
-  size_t len;
-  std::tie(p, len) = b.get();
-  CU_ASSERT(13 == len);
-  CU_ASSERT(0 == memcmp(p, "0123456789ABC", len));
-
-  b.drain(14);
-
-  CU_ASSERT(2 == b.rleft());
-  CU_ASSERT(14 == b.wleft());
-  CU_ASSERT(1 == b.pos);
-  CU_ASSERT(2 == b.len);
-
-  std::tie(p, len) = b.get();
-  CU_ASSERT(2 == len);
-  CU_ASSERT(0 == memcmp(p, "EF", len));
-}
-
-void test_ringbuf_iovec(void) {
-  RingBuf<16> b;
-  struct iovec iov[2];
-
-  auto rv = b.riovec(iov);
-
-  CU_ASSERT(0 == rv);
-
-  rv = b.wiovec(iov);
-
-  CU_ASSERT(1 == rv);
-  CU_ASSERT(b.begin == iov[0].iov_base);
-  CU_ASSERT(16 == iov[0].iov_len);
-
-  // set pos to somewhere middle of the buffer, this will require 2
-  // iovec for writing.
-  b.pos = 6;
-
-  rv = b.riovec(iov);
-
-  CU_ASSERT(0 == rv);
-
-  rv = b.wiovec(iov);
-
-  CU_ASSERT(2 == rv);
-  CU_ASSERT(b.begin + b.pos == iov[0].iov_base);
-  CU_ASSERT(10 == iov[0].iov_len);
-  CU_ASSERT(b.begin == iov[1].iov_base);
-  CU_ASSERT(6 == iov[1].iov_len);
-
-  // occupy first region of buffer
-  b.pos = 0;
-  b.len = 10;
-
-  rv = b.riovec(iov);
-
-  CU_ASSERT(1 == rv);
-  CU_ASSERT(b.begin == iov[0].iov_base);
-  CU_ASSERT(10 == iov[0].iov_len);
-
-  rv = b.wiovec(iov);
-
-  CU_ASSERT(1 == rv);
-  CU_ASSERT(b.begin + b.len == iov[0].iov_base);
-  CU_ASSERT(6 == iov[0].iov_len);
-
-  // occupy last region of buffer
-  b.pos = 6;
-  b.len = 10;
-
-  rv = b.riovec(iov);
-
-  CU_ASSERT(1 == rv);
-  CU_ASSERT(b.begin + b.pos == iov[0].iov_base);
-  CU_ASSERT(10 == iov[0].iov_len);
-
-  rv = b.wiovec(iov);
-
-  CU_ASSERT(1 == rv);
-  CU_ASSERT(b.begin == iov[0].iov_base);
-  CU_ASSERT(6 == iov[0].iov_len);
-
-  // occupy middle of buffer
-  b.pos = 3;
-  b.len = 10;
-
-  rv = b.riovec(iov);
-
-  CU_ASSERT(1 == rv);
-  CU_ASSERT(b.begin + b.pos == iov[0].iov_base);
-  CU_ASSERT(10 == iov[0].iov_len);
-
-  rv = b.wiovec(iov);
-
-  CU_ASSERT(2 == rv);
-  CU_ASSERT(b.begin + b.pos + b.len == iov[0].iov_base);
-  CU_ASSERT(3 == iov[0].iov_len);
-  CU_ASSERT(b.begin == iov[1].iov_base);
-  CU_ASSERT(3 == iov[1].iov_len);
-
-  // crossover
-  b.pos = 13;
-  b.len = 10;
-
-  rv = b.riovec(iov);
-
-  CU_ASSERT(2 == rv);
-  CU_ASSERT(b.begin + b.pos == iov[0].iov_base);
-  CU_ASSERT(3 == iov[0].iov_len);
-  CU_ASSERT(b.begin == iov[1].iov_base);
-  CU_ASSERT(7 == iov[1].iov_len);
-
-  rv = b.wiovec(iov);
-
-  CU_ASSERT(1 == rv);
-  CU_ASSERT(b.begin + 7 == iov[0].iov_base);
-  CU_ASSERT(6 == iov[0].iov_len);
-}
-
-} // namespace nghttp2

src/shrpx-unittest.cc

@@ -38,7 +38,7 @@
 #include "http2_test.h"
 #include "util_test.h"
 #include "nghttp2_gzip_test.h"
-#include "ringbuf_test.h"
+#include "buffer_test.h"
 #include "memchunk_test.h"
 #include "shrpx_config.h"
 
@@ -93,8 +93,9 @@ int main(int argc, char *argv[]) {
                    shrpx::test_http2_http2_header_allowed) ||
       !CU_add_test(pSuite, "http2_mandatory_request_headers_presence",
                    shrpx::test_http2_mandatory_request_headers_presence) ||
-      !CU_add_test(pSuite, "http2_check_http2_te",
-                   shrpx::test_http2_check_http2_te) ||
+      !CU_add_test(pSuite, "http2_parse_link_header",
+                   shrpx::test_http2_parse_link_header) ||
+      !CU_add_test(pSuite, "http2_path_join", shrpx::test_http2_path_join) ||
       !CU_add_test(pSuite, "downstream_index_request_headers",
                    shrpx::test_downstream_index_request_headers) ||
       !CU_add_test(pSuite, "downstream_index_response_headers",
@@ -122,8 +123,11 @@ int main(int argc, char *argv[]) {
       !CU_add_test(pSuite, "util_inp_strlower",
                    shrpx::test_util_inp_strlower) ||
       !CU_add_test(pSuite, "util_to_base64", shrpx::test_util_to_base64) ||
+      !CU_add_test(pSuite, "util_to_token68", shrpx::test_util_to_token68) ||
       !CU_add_test(pSuite, "util_percent_encode_token",
                    shrpx::test_util_percent_encode_token) ||
+      !CU_add_test(pSuite, "util_percent_decode",
+                   shrpx::test_util_percent_decode) ||
       !CU_add_test(pSuite, "util_quote_string",
                    shrpx::test_util_quote_string) ||
       !CU_add_test(pSuite, "util_utox", shrpx::test_util_utox) ||
@@ -133,12 +137,21 @@ int main(int argc, char *argv[]) {
                    shrpx::test_util_ipv6_numeric_addr) ||
       !CU_add_test(pSuite, "util_utos_with_unit",
                    shrpx::test_util_utos_with_unit) ||
+      !CU_add_test(pSuite, "util_utos_with_funit",
+                   shrpx::test_util_utos_with_funit) ||
       !CU_add_test(pSuite, "util_parse_uint_with_unit",
                    shrpx::test_util_parse_uint_with_unit) ||
       !CU_add_test(pSuite, "util_parse_uint", shrpx::test_util_parse_uint) ||
+      !CU_add_test(pSuite, "util_parse_duration_with_unit",
+                   shrpx::test_util_parse_duration_with_unit) ||
+      !CU_add_test(pSuite, "util_duration_str",
+                   shrpx::test_util_duration_str) ||
+      !CU_add_test(pSuite, "util_format_duration",
+                   shrpx::test_util_format_duration) ||
+      !CU_add_test(pSuite, "util_starts_with", shrpx::test_util_starts_with) ||
+      !CU_add_test(pSuite, "util_ends_with", shrpx::test_util_ends_with) ||
       !CU_add_test(pSuite, "gzip_inflate", test_nghttp2_gzip_inflate) ||
-      !CU_add_test(pSuite, "ringbuf_write", nghttp2::test_ringbuf_write) ||
-      !CU_add_test(pSuite, "ringbuf_iovec", nghttp2::test_ringbuf_iovec) ||
+      !CU_add_test(pSuite, "buffer_write", nghttp2::test_buffer_write) ||
       !CU_add_test(pSuite, "pool_recycle", nghttp2::test_pool_recycle) ||
       !CU_add_test(pSuite, "memchunk_append", nghttp2::test_memchunks_append) ||
       !CU_add_test(pSuite, "memchunk_drain", nghttp2::test_memchunks_drain) ||

src/shrpx.cc

@@ -28,9 +28,11 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <sys/socket.h>
+#include <sys/un.h>
 #include <netdb.h>
 #include <signal.h>
 #include <netinet/in.h>
+#include <netinet/tcp.h>
 #include <arpa/inet.h>
 #include <unistd.h>
 #include <getopt.h>
@@ -39,12 +41,14 @@
 #include <limits.h>
 #include <sys/time.h>
 #include <sys/resource.h>
+#include <grp.h>
 
 #include <limits>
 #include <cstdlib>
 #include <iostream>
 #include <fstream>
 #include <vector>
+#include <initializer_list>
 
 #include <openssl/ssl.h>
 #include <openssl/err.h>
@@ -58,12 +62,15 @@
 #include "shrpx_config.h"
 #include "shrpx_connection_handler.h"
 #include "shrpx_ssl.h"
-#include "shrpx_worker_config.h"
+#include "shrpx_log_config.h"
 #include "shrpx_worker.h"
 #include "shrpx_accept_handler.h"
+#include "shrpx_http2_upstream.h"
+#include "shrpx_http2_session.h"
 #include "util.h"
 #include "app_helper.h"
 #include "ssl.h"
+#include "template.h"
 
 extern char **environ;
 
@@ -86,6 +93,13 @@ const int GRACEFUL_SHUTDOWN_SIGNAL = SIGQUIT;
 // binary is listening to.
 #define ENV_PORT "NGHTTPX_PORT"
 
+// Environment variable to tell new binary the listening socket's file
+// descriptor if frontend listens UNIX domain socket.
+#define ENV_UNIX_FD "NGHTTP2_UNIX_FD"
+// Environment variable to tell new binary the UNIX domain socket
+// path.
+#define ENV_UNIX_PATH "NGHTTP2_UNIX_PATH"
+
 namespace {
 int resolve_hostname(sockaddr_union *addr, size_t *addrlen,
                      const char *hostname, uint16_t port, int family) {
@@ -133,6 +147,95 @@ int resolve_hostname(sockaddr_union *addr, size_t *addrlen,
 }
 } // namespace
 
+namespace {
+void close_env_fd(std::initializer_list<const char *> envnames) {
+  for (auto envname : envnames) {
+    auto envfd = getenv(envname);
+    if (!envfd) {
+      continue;
+    }
+    auto fd = strtol(envfd, nullptr, 10);
+    close(fd);
+  }
+}
+} // namespace
+
+namespace {
+std::unique_ptr<AcceptHandler>
+create_unix_domain_acceptor(ConnectionHandler *handler) {
+  auto path = get_config()->host.get();
+  auto pathlen = strlen(path);
+  {
+    auto envfd = getenv(ENV_UNIX_FD);
+    auto envpath = getenv(ENV_UNIX_PATH);
+    if (envfd && envpath) {
+      auto fd = strtoul(envfd, nullptr, 10);
+
+      if (util::streq(envpath, path)) {
+        LOG(NOTICE) << "Listening on UNIX domain socket " << path;
+
+        return make_unique<AcceptHandler>(fd, handler);
+      }
+
+      LOG(WARN) << "UNIX domain socket path was changed between old binary ("
+                << envpath << ") and new binary (" << path << ")";
+      close(fd);
+    }
+  }
+
+#ifdef SOCK_NONBLOCK
+  auto fd = socket(AF_UNIX, SOCK_STREAM | SOCK_NONBLOCK, 0);
+  if (fd == -1) {
+    return nullptr;
+  }
+#else  // !SOCK_NONBLOCK
+  auto fd = socket(AF_UNIX, SOCK_STREAM, 0);
+  if (fd == -1) {
+    return nullptr;
+  }
+  util::make_socket_nonblocking(fd);
+#endif // !SOCK_NONBLOCK
+  int val = 1;
+  if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &val,
+                 static_cast<socklen_t>(sizeof(val))) == -1) {
+    close(fd);
+    return nullptr;
+  }
+
+  sockaddr_union addr;
+  addr.un.sun_family = AF_UNIX;
+  if (pathlen + 1 > sizeof(addr.un.sun_path)) {
+    LOG(FATAL) << "UNIX domain socket path " << path << " is too long > "
+               << sizeof(addr.un.sun_path);
+    close(fd);
+    return nullptr;
+  }
+  // copy path including terminal NULL
+  std::copy_n(path, pathlen + 1, addr.un.sun_path);
+
+  // unlink (remove) already existing UNIX domain socket path
+  unlink(path);
+
+  if (bind(fd, &addr.sa, sizeof(addr.un)) != 0) {
+    auto error = errno;
+    LOG(FATAL) << "Failed to bind UNIX domain socket, error=" << error;
+    close(fd);
+    return nullptr;
+  }
+
+  if (listen(fd, get_config()->backlog) != 0) {
+    auto error = errno;
+    LOG(FATAL) << "Failed to listen to UNIX domain socket, error=" << error;
+    close(fd);
+    return nullptr;
+  }
+
+  LOG(NOTICE) << "Listening on UNIX domain socket " << path;
+
+  return make_unique<AcceptHandler>(fd, handler);
+}
+} // namespace
+
 namespace {
 std::unique_ptr<AcceptHandler> create_acceptor(ConnectionHandler *handler,
                                                int family) {
@@ -151,7 +254,7 @@ std::unique_ptr<AcceptHandler> create_acceptor(ConnectionHandler *handler,
       if (port == get_config()->port) {
         LOG(NOTICE) << "Listening on port " << get_config()->port;
 
-        return util::make_unique<AcceptHandler>(fd, handler);
+        return make_unique<AcceptHandler>(fd, handler);
       }
 
       LOG(WARN) << "Port was changed between old binary (" << port
@@ -217,6 +320,15 @@ std::unique_ptr<AcceptHandler> create_acceptor(ConnectionHandler *handler,
       }
     }
 #endif // IPV6_V6ONLY
+
+#ifdef TCP_DEFER_ACCEPT
+    val = 3;
+    if (setsockopt(fd, IPPROTO_TCP, TCP_DEFER_ACCEPT, &val,
+                   static_cast<socklen_t>(sizeof(val))) == -1) {
+      LOG(WARN) << "Failed to set TCP_DEFER_ACCEPT option to listener socket";
+    }
+#endif // TCP_DEFER_ACCEPT
+
     if (bind(fd, rp->ai_addr, rp->ai_addrlen) == 0 &&
         listen(fd, get_config()->backlog) == 0) {
       break;
@@ -249,13 +361,19 @@ std::unique_ptr<AcceptHandler> create_acceptor(ConnectionHandler *handler,
 
   LOG(NOTICE) << "Listening on " << host << ", port " << get_config()->port;
 
-  return util::make_unique<AcceptHandler>(fd, handler);
+  return make_unique<AcceptHandler>(fd, handler);
 }
 } // namespace
 
 namespace {
 void drop_privileges() {
   if (getuid() == 0 && get_config()->uid != 0) {
+    if (initgroups(get_config()->user.get(), get_config()->gid) != 0) {
+      auto error = errno;
+      LOG(FATAL) << "Could not change supplementary groups: "
+                 << strerror(error);
+      exit(EXIT_FAILURE);
+    }
     if (setgid(get_config()->gid) != 0) {
       auto error = errno;
       LOG(FATAL) << "Could not change gid: " << strerror(error);
@@ -300,7 +418,7 @@ void reopen_log_signal_cb(struct ev_loop *loop, ev_signal *w, int revents) {
   auto conn_handler = static_cast<ConnectionHandler *>(w->data);
 
   if (LOG_ENABLED(INFO)) {
-    LOG(INFO) << "Reopening log files: worker_info(" << worker_config << ")";
+    LOG(INFO) << "Reopening log files: main";
   }
 
   (void)reopen_log_files();
@@ -337,7 +455,7 @@ void exec_binary_signal_cb(struct ev_loop *loop, ev_signal *w, int revents) {
     return;
   }
 
-  auto argv = util::make_unique<char *[]>(get_config()->argc + 1);
+  auto argv = make_unique<char *[]>(get_config()->argc + 1);
 
   argv[0] = exec_path;
   for (int i = 1; i < get_config()->argc; ++i) {
@@ -348,32 +466,45 @@ void exec_binary_signal_cb(struct ev_loop *loop, ev_signal *w, int revents) {
   size_t envlen = 0;
   for (char **p = environ; *p; ++p, ++envlen)
     ;
-  // 3 for missing fd4, fd6 and port.
-  auto envp = util::make_unique<char *[]>(envlen + 3 + 1);
+  // 3 for missing (fd4, fd6 and port) or (unix fd and unix path)
+  auto envp = make_unique<char *[]>(envlen + 3 + 1);
   size_t envidx = 0;
 
-  auto acceptor4 = conn_handler->get_acceptor4();
-  if (acceptor4) {
-    std::string fd4 = ENV_LISTENER4_FD "=";
-    fd4 += util::utos(acceptor4->get_fd());
-    envp[envidx++] = strdup(fd4.c_str());
-  }
+  if (get_config()->host_unix) {
+    auto acceptor = conn_handler->get_acceptor();
+    std::string fd = ENV_UNIX_FD "=";
+    fd += util::utos(acceptor->get_fd());
+    envp[envidx++] = strdup(fd.c_str());
 
-  auto acceptor6 = conn_handler->get_acceptor6();
-  if (acceptor6) {
-    std::string fd6 = ENV_LISTENER6_FD "=";
-    fd6 += util::utos(acceptor6->get_fd());
-    envp[envidx++] = strdup(fd6.c_str());
-  }
+    std::string path = ENV_UNIX_PATH "=";
+    path += get_config()->host.get();
+    envp[envidx++] = strdup(path.c_str());
+  } else {
+    auto acceptor4 = conn_handler->get_acceptor();
+    if (acceptor4) {
+      std::string fd4 = ENV_LISTENER4_FD "=";
+      fd4 += util::utos(acceptor4->get_fd());
+      envp[envidx++] = strdup(fd4.c_str());
+    }
 
-  std::string port = ENV_PORT "=";
-  port += util::utos(get_config()->port);
-  envp[envidx++] = strdup(port.c_str());
+    auto acceptor6 = conn_handler->get_acceptor6();
+    if (acceptor6) {
+      std::string fd6 = ENV_LISTENER6_FD "=";
+      fd6 += util::utos(acceptor6->get_fd());
+      envp[envidx++] = strdup(fd6.c_str());
+    }
+
+    std::string port = ENV_PORT "=";
+    port += util::utos(get_config()->port);
+    envp[envidx++] = strdup(port.c_str());
+  }
 
   for (size_t i = 0; i < envlen; ++i) {
-    if (strcmp(ENV_LISTENER4_FD, environ[i]) == 0 ||
-        strcmp(ENV_LISTENER6_FD, environ[i]) == 0 ||
-        strcmp(ENV_PORT, environ[i]) == 0) {
+    if (util::startsWith(environ[i], ENV_LISTENER4_FD) ||
+        util::startsWith(environ[i], ENV_LISTENER6_FD) ||
+        util::startsWith(environ[i], ENV_PORT) ||
+        util::startsWith(environ[i], ENV_UNIX_FD) ||
+        util::startsWith(environ[i], ENV_UNIX_PATH)) {
       continue;
     }
 
@@ -406,13 +537,13 @@ void graceful_shutdown_signal_cb(struct ev_loop *loop, ev_signal *w,
                                  int revents) {
   auto conn_handler = static_cast<ConnectionHandler *>(w->data);
 
-  if (worker_config->graceful_shutdown) {
+  if (conn_handler->get_graceful_shutdown()) {
     return;
   }
 
   LOG(NOTICE) << "Graceful shutdown signal received";
 
-  worker_config->graceful_shutdown = true;
+  conn_handler->set_graceful_shutdown(true);
 
   conn_handler->disable_acceptor();
 
@@ -436,12 +567,12 @@ void graceful_shutdown_signal_cb(struct ev_loop *loop, ev_signal *w,
 namespace {
 void refresh_cb(struct ev_loop *loop, ev_timer *w, int revents) {
   auto conn_handler = static_cast<ConnectionHandler *>(w->data);
-  auto worker_stat = conn_handler->get_worker_stat();
+  auto worker = conn_handler->get_single_worker();
 
   // In multi threaded mode (get_config()->num_worker > 1), we have to
   // wait for event notification to workers to finish.
-  if (get_config()->num_worker == 1 && worker_config->graceful_shutdown &&
-      (!worker_stat || worker_stat->num_connections == 0)) {
+  if (get_config()->num_worker == 1 && conn_handler->get_graceful_shutdown() &&
+      (!worker || worker->get_worker_stat()->num_connections == 0)) {
     ev_break(loop);
   }
 }
@@ -450,7 +581,7 @@ void refresh_cb(struct ev_loop *loop, ev_timer *w, int revents) {
 namespace {
 void renew_ticket_key_cb(struct ev_loop *loop, ev_timer *w, int revents) {
   auto conn_handler = static_cast<ConnectionHandler *>(w->data);
-  const auto &old_ticket_keys = worker_config->ticket_keys;
+  const auto &old_ticket_keys = conn_handler->get_ticket_keys();
 
   auto ticket_keys = std::make_shared<TicketKeys>();
   if (LOG_ENABLED(INFO)) {
@@ -484,8 +615,7 @@ void renew_ticket_key_cb(struct ev_loop *loop, ev_timer *w, int revents) {
     }
   }
 
-  worker_config->ticket_keys = ticket_keys;
-
+  conn_handler->set_ticket_keys(ticket_keys);
   conn_handler->worker_renew_ticket_keys(ticket_keys);
 }
 } // namespace
@@ -494,7 +624,7 @@ namespace {
 int event_loop() {
   auto loop = EV_DEFAULT;
 
-  auto conn_handler = util::make_unique<ConnectionHandler>(loop);
+  auto conn_handler = make_unique<ConnectionHandler>(loop);
   if (get_config()->daemon) {
     if (daemon(0, 0) == -1) {
       auto error = errno;
@@ -510,33 +640,59 @@ int event_loop() {
     save_pid();
   }
 
-  auto acceptor6 = create_acceptor(conn_handler.get(), AF_INET6);
-  auto acceptor4 = create_acceptor(conn_handler.get(), AF_INET);
-  if (!acceptor6 && !acceptor4) {
-    LOG(FATAL) << "Failed to listen on address " << get_config()->host.get()
-               << ", port " << get_config()->port;
-    exit(EXIT_FAILURE);
+  if (get_config()->host_unix) {
+    close_env_fd({ENV_LISTENER4_FD, ENV_LISTENER6_FD});
+    auto acceptor = create_unix_domain_acceptor(conn_handler.get());
+    if (!acceptor) {
+      LOG(FATAL) << "Failed to listen on UNIX domain socket "
+                 << get_config()->host.get();
+      exit(EXIT_FAILURE);
+    }
+
+    conn_handler->set_acceptor(std::move(acceptor));
+  } else {
+    close_env_fd({ENV_UNIX_FD});
+    auto acceptor6 = create_acceptor(conn_handler.get(), AF_INET6);
+    auto acceptor4 = create_acceptor(conn_handler.get(), AF_INET);
+    if (!acceptor6 && !acceptor4) {
+      LOG(FATAL) << "Failed to listen on address " << get_config()->host.get()
+                 << ", port " << get_config()->port;
+      exit(EXIT_FAILURE);
+    }
+
+    conn_handler->set_acceptor(std::move(acceptor4));
+    conn_handler->set_acceptor6(std::move(acceptor6));
   }
 
-  conn_handler->set_acceptor4(std::move(acceptor4));
-  conn_handler->set_acceptor6(std::move(acceptor6));
+  ev_timer renew_ticket_key_timer;
+  if (!get_config()->upstream_no_tls) {
+    bool auto_tls_ticket_key = true;
+    if (!get_config()->tls_ticket_key_files.empty()) {
+      auto ticket_keys =
+          read_tls_ticket_key_file(get_config()->tls_ticket_key_files);
+      if (!ticket_keys) {
+        LOG(WARN) << "Use internal session ticket key generator";
+      } else {
+        conn_handler->set_ticket_keys(std::move(ticket_keys));
+        auto_tls_ticket_key = false;
+      }
+    }
+    if (auto_tls_ticket_key) {
+      // Renew ticket key every 12hrs
+      ev_timer_init(&renew_ticket_key_timer, renew_ticket_key_cb, 0.,
+                    12 * 3600.);
+      renew_ticket_key_timer.data = conn_handler.get();
+      ev_timer_again(loop, &renew_ticket_key_timer);
+
+      // Generate first session ticket key before running workers.
+      renew_ticket_key_cb(loop, &renew_ticket_key_timer, 0);
+    }
+  }
 
   // ListenHandler loads private key, and we listen on a priveleged port.
   // After that, we drop the root privileges if needed.
   drop_privileges();
 
-  ev_timer renew_ticket_key_timer;
-  if (!get_config()->client_mode && !get_config()->upstream_no_tls &&
-      get_config()->auto_tls_ticket_key) {
-    // Renew ticket key every 12hrs
-    ev_timer_init(&renew_ticket_key_timer, renew_ticket_key_cb, 0., 12 * 3600.);
-    renew_ticket_key_timer.data = conn_handler.get();
-    ev_timer_again(loop, &renew_ticket_key_timer);
-
-    // Generate first session ticket key before running workers.
-    renew_ticket_key_cb(loop, &renew_ticket_key_timer, 0);
-  }
-
 #ifndef NOTHREADS
   int rv;
   sigset_t signals;
@@ -550,18 +706,10 @@ int event_loop() {
   }
 #endif // !NOTHREADS
 
-  if (get_config()->num_worker > 1) {
-    if (!get_config()->tls_ctx_per_worker) {
-      conn_handler->create_ssl_context();
-    }
-    conn_handler->create_worker_thread(get_config()->num_worker);
+  if (get_config()->num_worker == 1) {
+    conn_handler->create_single_worker();
   } else {
-    conn_handler->create_ssl_context();
-    if (get_config()->downstream_proto == PROTO_HTTP2) {
-      conn_handler->create_http2_session();
-    } else {
-      conn_handler->create_http1_connect_blocker();
-    }
+    conn_handler->create_worker_thread(get_config()->num_worker);
   }
 
 #ifndef NOTHREADS
@@ -614,7 +762,7 @@ bool conf_exists(const char *path) {
 } // namespace
 
 namespace {
-const char *DEFAULT_NPN_LIST = "h2-16," NGHTTP2_PROTO_VERSION_ID ","
+const char *DEFAULT_NPN_LIST = "h2,h2-16," NGHTTP2_PROTO_VERSION_ID ","
 #ifdef HAVE_SPDYLAY
                                "spdy/3.1,"
 #endif // HAVE_SPDYLAY
@@ -670,7 +818,7 @@ void fill_default_config() {
   mod_config()->stream_write_timeout = 0.;
 
   // Timeout for pooled (idle) connections
-  mod_config()->downstream_idle_read_timeout = 600.;
+  mod_config()->downstream_idle_read_timeout = 2.;
 
   // window bits for HTTP/2 and SPDY upstream/downstream connection
   // per stream. 2**16-1 = 64KiB-1, which is HTTP/2 default. Please
@@ -714,6 +862,7 @@ void fill_default_config() {
   mod_config()->insecure = false;
   mod_config()->cacert = nullptr;
   mod_config()->pid_file = nullptr;
+  mod_config()->user = nullptr;
   mod_config()->uid = 0;
   mod_config()->gid = 0;
   mod_config()->pid = getpid();
@@ -742,21 +891,32 @@ void fill_default_config() {
   mod_config()->padding = 0;
   mod_config()->worker_frontend_connections = 0;
 
-  nghttp2_option_new(&mod_config()->http2_option);
+  mod_config()->http2_upstream_callbacks = create_http2_upstream_callbacks();
+  mod_config()->http2_downstream_callbacks =
+      create_http2_downstream_callbacks();
 
-  nghttp2_option_set_no_auto_window_update(mod_config()->http2_option, 1);
+  nghttp2_option_new(&mod_config()->http2_option);
+  nghttp2_option_set_no_auto_window_update(get_config()->http2_option, 1);
+
+  nghttp2_option_new(&mod_config()->http2_client_option);
+  nghttp2_option_set_no_auto_window_update(get_config()->http2_client_option,
+                                           1);
+  nghttp2_option_set_peer_max_concurrent_streams(
+      get_config()->http2_client_option, 100);
 
   mod_config()->tls_proto_mask = 0;
   mod_config()->no_location_rewrite = false;
+  mod_config()->no_host_rewrite = false;
   mod_config()->argc = 0;
   mod_config()->argv = nullptr;
   mod_config()->downstream_connections_per_host = 8;
   mod_config()->downstream_connections_per_frontend = 0;
   mod_config()->listener_disable_timeout = 0.;
-  mod_config()->auto_tls_ticket_key = true;
   mod_config()->tls_ctx_per_worker = false;
   mod_config()->downstream_request_buffer_size = 16 * 1024;
   mod_config()->downstream_response_buffer_size = 16 * 1024;
+  mod_config()->no_server_push = false;
+  mod_config()->host_unix = false;
 }
 } // namespace
 
@@ -792,12 +952,16 @@ Connections:
               backend addresses are accepted by repeating this option.
               HTTP/2  backend   does  not  support   multiple  backend
               addresses  and the  first occurrence  of this  option is
-              used.
+              used.  UNIX domain socket  can be specified by prefixing
+              path        name        with       "unix:"        (e.g.,
+              unix:/var/run/backend.sock)
               Default: )" << DEFAULT_DOWNSTREAM_HOST << ","
       << DEFAULT_DOWNSTREAM_PORT << R"(
   -f, --frontend=<HOST,PORT>
               Set  frontend  host and  port.   If  <HOST> is  '*',  it
-              assumes all addresses including both IPv4 and IPv6.
+              assumes  all addresses  including  both  IPv4 and  IPv6.
+              UNIX domain  socket can  be specified by  prefixing path
+              name with "unix:" (e.g., unix:/var/run/nghttpx.sock)
               Default: )" << get_config()->host.get() << ","
       << get_config()->port << R"(
   --backlog=<N>
@@ -899,38 +1063,47 @@ Performance:
       << R"(
 
 Timeout:
-  --frontend-http2-read-timeout=<SEC>
+  --frontend-http2-read-timeout=<DURATION>
               Specify  read  timeout  for  HTTP/2  and  SPDY  frontend
               connection.
-              Default: )" << get_config()->http2_upstream_read_timeout << R"(
-  --frontend-read-timeout=<SEC>
+              Default: )"
+      << util::duration_str(get_config()->http2_upstream_read_timeout) << R"(
+  --frontend-read-timeout=<DURATION>
               Specify read timeout for HTTP/1.1 frontend connection.
-              Default: )" << get_config()->upstream_read_timeout << R"(
-  --frontend-write-timeout=<SEC>
+              Default: )"
+      << util::duration_str(get_config()->upstream_read_timeout) << R"(
+  --frontend-write-timeout=<DURATION>
               Specify write timeout for all frontend connections.
-              Default: )" << get_config()->upstream_write_timeout << R"(
-  --stream-read-timeout=<SEC>
+              Default: )"
+      << util::duration_str(get_config()->upstream_write_timeout) << R"(
+  --stream-read-timeout=<DURATION>
               Specify  read timeout  for HTTP/2  and SPDY  streams.  0
               means no timeout.
-              Default: )" << get_config()->stream_read_timeout << R"(
-  --stream-write-timeout=<SEC>
+              Default: )"
+      << util::duration_str(get_config()->stream_read_timeout) << R"(
+  --stream-write-timeout=<DURATION>
               Specify write  timeout for  HTTP/2 and SPDY  streams.  0
               means no timeout.
-              Default: )" << get_config()->stream_write_timeout << R"(
-  --backend-read-timeout=<SEC>
+              Default: )"
+      << util::duration_str(get_config()->stream_write_timeout) << R"(
+  --backend-read-timeout=<DURATION>
               Specify read timeout for backend connection.
-              Default: )" << get_config()->downstream_read_timeout << R"(
-  --backend-write-timeout=<SEC>
+              Default: )"
+      << util::duration_str(get_config()->downstream_read_timeout) << R"(
+  --backend-write-timeout=<DURATION>
               Specify write timeout for backend connection.
-              Default: )" << get_config()->downstream_write_timeout << R"(
-  --backend-keep-alive-timeout=<SEC>
+              Default: )"
+      << util::duration_str(get_config()->downstream_write_timeout) << R"(
+  --backend-keep-alive-timeout=<DURATION>
               Specify keep-alive timeout for backend connection.
-              Default: )" << get_config()->downstream_idle_read_timeout << R"(
-  --listener-disable-timeout=<SEC>
+              Default: )"
+      << util::duration_str(get_config()->downstream_idle_read_timeout) << R"(
+  --listener-disable-timeout=<DURATION>
               After accepting  connection failed,  connection listener
-              is disabled for  a given time in  seconds.  Specifying 0
+              is disabled  for a given  amount of time.   Specifying 0
               disables this feature.
-              Default: )" << get_config()->listener_disable_timeout << R"(
+              Default: )"
+      << util::duration_str(get_config()->listener_disable_timeout) << R"(
 
 SSL/TLS:
   --ciphers=<SUITE>
@@ -1053,6 +1226,10 @@ HTTP/2 and SPDY:
               padding.  Specify 0 to  disable padding.  This option is
               meant for debugging purpose  and not intended to enhance
               protocol security.
+  --no-server-push
+              Disable  HTTP/2  server  push.    Server  push  is  only
+              supported  by default  mode and  HTTP/2 frontend.   SPDY
+              frontend does not support server push.
 
 Mode:
   (default mode)
@@ -1141,6 +1318,11 @@ HTTP:
               --client  and  default   mode.   For  --http2-proxy  and
               --client-proxy mode,  location header field will  not be
               altered regardless of this option.
+  --no-host-rewrite
+              Don't  rewrite  host  and :authority  header  fields  on
+              --http2-bridge,   --client   and  default   mode.    For
+              --http2-proxy  and  --client-proxy mode,  these  headers
+              will not be altered regardless of this option.
   --altsvc=<PROTOID,PORT[,HOST,[ORIGIN]]>
               Specify   protocol  ID,   port,  host   and  origin   of
               alternative service.  <HOST>  and <ORIGIN> are optional.
@@ -1192,7 +1374,11 @@ Misc:
   -h, --help  Print this help and exit.
 
   The <SIZE> argument is an integer and an optional unit (e.g., 10K is
-  10 * 1024).  Units are K, M and G (powers of 1024).)" << std::endl;
+  10 * 1024).  Units are K, M and G (powers of 1024).
+
+  The <DURATION> argument is an integer and an optional unit (e.g., 1s
+  is 1 second and 500ms is 500  milliseconds).  Units are s or ms.  If
+  a unit is omitted, a second is used as unit.)" << std::endl;
 }
 } // namespace
 
@@ -1301,6 +1487,8 @@ int main(int argc, char **argv) {
         {"tls-ctx-per-worker", no_argument, &flag, 70},
         {"backend-response-buffer", required_argument, &flag, 71},
         {"backend-request-buffer", required_argument, &flag, 72},
+        {"no-host-rewrite", no_argument, &flag, 73},
+        {"no-server-push", no_argument, &flag, 74},
         {nullptr, 0, nullptr, 0}};
 
     int option_index = 0;
@@ -1630,6 +1818,14 @@ int main(int argc, char **argv) {
         // --backend-request-buffer
         cmdcfgs.emplace_back(SHRPX_OPT_BACKEND_REQUEST_BUFFER, optarg);
         break;
+      case 73:
+        // --no-host-rewrite
+        cmdcfgs.emplace_back(SHRPX_OPT_NO_HOST_REWRITE, "yes");
+        break;
+      case 74:
+        // --no-server-push
+        cmdcfgs.emplace_back(SHRPX_OPT_NO_SERVER_PUSH, "yes");
+        break;
       default:
         break;
       }
@@ -1673,7 +1869,7 @@ int main(int argc, char **argv) {
 #ifndef NOTHREADS
   std::unique_ptr<nghttp2::ssl::LibsslGlobalLock> lock;
   if (!get_config()->tls_ctx_per_worker) {
-    lock = util::make_unique<nghttp2::ssl::LibsslGlobalLock>();
+    lock = make_unique<nghttp2::ssl::LibsslGlobalLock>();
   }
 #endif // NOTHREADS
 
@@ -1688,16 +1884,16 @@ int main(int argc, char **argv) {
   }
 
   if (get_config()->uid != 0) {
-    if (worker_config->accesslog_fd != -1 &&
-        fchown(worker_config->accesslog_fd, get_config()->uid,
+    if (log_config->accesslog_fd != -1 &&
+        fchown(log_config->accesslog_fd, get_config()->uid,
                get_config()->gid) == -1) {
       auto error = errno;
       LOG(WARN) << "Changing owner of access log file failed: "
                 << strerror(error);
     }
-    if (worker_config->errorlog_fd != -1 &&
-        fchown(worker_config->errorlog_fd, get_config()->uid,
-               get_config()->gid) == -1) {
+    if (log_config->errorlog_fd != -1 &&
+        fchown(log_config->errorlog_fd, get_config()->uid, get_config()->gid) ==
+            -1) {
       auto error = errno;
       LOG(WARN) << "Changing owner of error log file failed: "
                 << strerror(error);
@@ -1759,17 +1955,6 @@ int main(int argc, char **argv) {
 
   mod_config()->alpn_prefs = ssl::set_alpn_prefs(get_config()->npn_list);
 
-  if (!get_config()->tls_ticket_key_files.empty()) {
-    auto ticket_keys =
-        read_tls_ticket_key_file(get_config()->tls_ticket_key_files);
-    if (!ticket_keys) {
-      LOG(WARN) << "Use internal session ticket key generator";
-    } else {
-      worker_config->ticket_keys = std::move(ticket_keys);
-      mod_config()->auto_tls_ticket_key = false;
-    }
-  }
-
   if (get_config()->backend_ipv4 && get_config()->backend_ipv6) {
     LOG(FATAL) << "--backend-ipv4 and --backend-ipv6 cannot be used at the "
                << "same time.";
@@ -1791,6 +1976,7 @@ int main(int argc, char **argv) {
 
   if (get_config()->client || get_config()->client_proxy) {
     mod_config()->client_mode = true;
+    mod_config()->upstream_no_tls = true;
   }
 
   if (get_config()->client_mode || get_config()->http2_bridge) {
@@ -1799,12 +1985,11 @@ int main(int argc, char **argv) {
     mod_config()->downstream_proto = PROTO_HTTP;
   }
 
-  if (!get_config()->client_mode && !get_config()->upstream_no_tls) {
-    if (!get_config()->private_key_file || !get_config()->cert_file) {
-      print_usage(std::cerr);
-      LOG(FATAL) << "Too few arguments";
-      exit(EXIT_FAILURE);
-    }
+  if (!get_config()->upstream_no_tls &&
+      (!get_config()->private_key_file || !get_config()->cert_file)) {
+    print_usage(std::cerr);
+    LOG(FATAL) << "Too few arguments";
+    exit(EXIT_FAILURE);
   }
 
   if (get_config()->downstream_addrs.empty()) {
@@ -1820,23 +2005,35 @@ int main(int argc, char **argv) {
   }
 
   for (auto &addr : mod_config()->downstream_addrs) {
-    auto ipv6 = util::ipv6_numeric_addr(addr.host.get());
-    std::string hostport;
 
-    if (ipv6) {
-      hostport += "[";
+    if (addr.host_unix) {
+      // for AF_UNIX socket, we use "localhost" as host for backend
+      // hostport.  This is used as Host header field to backend and
+      // not going to be passed to any syscalls.
+      addr.hostport =
+          strcopy(util::make_hostport("localhost", get_config()->port));
+
+      auto path = addr.host.get();
+      auto pathlen = strlen(path);
+
+      if (pathlen + 1 > sizeof(addr.addr.un.sun_path)) {
+        LOG(FATAL) << "UNIX domain socket path " << path << " is too long > "
+                   << sizeof(addr.addr.un.sun_path);
+        exit(EXIT_FAILURE);
+      }
+
+      LOG(INFO) << "Use UNIX domain socket path " << path
+                << " for backend connection";
+
+      addr.addr.un.sun_family = AF_UNIX;
+      // copy path including terminal NULL
+      std::copy_n(path, pathlen + 1, addr.addr.un.sun_path);
+      addr.addrlen = sizeof(addr.addr.un);
+
+      continue;
     }
 
-    hostport += addr.host.get();
-
-    if (ipv6) {
-      hostport += "]";
-    }
-
-    hostport += ":";
-    hostport += util::utos(addr.port);
-
-    addr.hostport = strcopy(hostport);
+    addr.hostport = strcopy(util::make_hostport(addr.host.get(), addr.port));
 
     if (resolve_hostname(
             &addr.addr, &addr.addrlen, addr.host.get(), addr.port,
@@ -1861,8 +2058,8 @@ int main(int argc, char **argv) {
   }
 
   if (get_config()->rlimit_nofile) {
-    struct rlimit lim = {get_config()->rlimit_nofile,
-                         get_config()->rlimit_nofile};
+    struct rlimit lim = {static_cast<rlim_t>(get_config()->rlimit_nofile),
+                         static_cast<rlim_t>(get_config()->rlimit_nofile)};
     if (setrlimit(RLIMIT_NOFILE, &lim) != 0) {
       auto error = errno;
       LOG(WARN) << "Setting rlimit-nofile failed: " << strerror(error);

src/shrpx_accept_handler.cc

@@ -83,7 +83,7 @@ void AcceptHandler::accept_connection() {
         continue;
       }
 
-      return;
+      break;
     }
 
 #ifndef HAVE_ACCEPT4

src/shrpx_client_handler.cc

@@ -35,13 +35,13 @@
 #include "shrpx_http2_downstream_connection.h"
 #include "shrpx_ssl.h"
 #include "shrpx_worker.h"
-#include "shrpx_worker_config.h"
 #include "shrpx_downstream_connection_pool.h"
 #include "shrpx_downstream.h"
 #ifdef HAVE_SPDYLAY
 #include "shrpx_spdy_upstream.h"
 #endif // HAVE_SPDYLAY
 #include "util.h"
+#include "template.h"
 
 using namespace nghttp2;
 
@@ -49,7 +49,8 @@ namespace shrpx {
 
 namespace {
 void timeoutcb(struct ev_loop *loop, ev_timer *w, int revents) {
-  auto handler = static_cast<ClientHandler *>(w->data);
+  auto conn = static_cast<Connection *>(w->data);
+  auto handler = static_cast<ClientHandler *>(conn->data);
 
   if (LOG_ENABLED(INFO)) {
     CLOG(INFO, handler) << "Time out";
@@ -73,18 +74,26 @@ void shutdowncb(struct ev_loop *loop, ev_timer *w, int revents) {
 
 namespace {
 void readcb(struct ev_loop *loop, ev_io *w, int revents) {
-  auto handler = static_cast<ClientHandler *>(w->data);
+  auto conn = static_cast<Connection *>(w->data);
+  auto handler = static_cast<ClientHandler *>(conn->data);
 
   if (handler->do_read() != 0) {
     delete handler;
     return;
   }
+  if (ev_is_active(handler->get_wev())) {
+    if (handler->do_write() != 0) {
+      delete handler;
+      return;
+    }
+  }
 }
 } // namespace
 
 namespace {
 void writecb(struct ev_loop *loop, ev_io *w, int revents) {
-  auto handler = static_cast<ClientHandler *>(w->data);
+  auto conn = static_cast<Connection *>(w->data);
+  auto handler = static_cast<ClientHandler *>(conn->data);
 
   if (handler->do_write() != 0) {
     delete handler;
@@ -94,70 +103,46 @@ void writecb(struct ev_loop *loop, ev_io *w, int revents) {
 } // namespace
 
 int ClientHandler::read_clear() {
-  ev_timer_again(loop_, &rt_);
+  ev_timer_again(conn_.loop, &conn_.rt);
 
   for (;;) {
-    // we should process buffered data first before we read EOF.
     if (rb_.rleft() && on_read() != 0) {
       return -1;
     }
-    if (rb_.rleft()) {
+    if (rb_.rleft() == 0) {
+      rb_.reset();
+    } else if (rb_.wleft() == 0) {
+      conn_.rlimit.stopw();
       return 0;
     }
-    rb_.reset();
-    struct iovec iov[2];
-    auto iovcnt = rb_.wiovec(iov);
-    iovcnt = limit_iovec(iov, iovcnt, rlimit_.avail());
-    if (iovcnt == 0) {
-      break;
-    }
 
-    ssize_t nread;
-    while ((nread = readv(fd_, iov, iovcnt)) == -1 && errno == EINTR)
-      ;
-    if (nread == -1) {
-      if (errno == EAGAIN || errno == EWOULDBLOCK) {
-        break;
-      }
-      return -1;
-    }
+    auto nread = conn_.read_clear(rb_.last, rb_.wleft());
 
     if (nread == 0) {
+      return 0;
+    }
+
+    if (nread < 0) {
       return -1;
     }
 
     rb_.write(nread);
-    rlimit_.drain(nread);
   }
-
-  return 0;
 }
 
 int ClientHandler::write_clear() {
-  ev_timer_again(loop_, &rt_);
+  ev_timer_again(conn_.loop, &conn_.rt);
 
   for (;;) {
     if (wb_.rleft() > 0) {
-      struct iovec iov[2];
-      auto iovcnt = wb_.riovec(iov);
-      iovcnt = limit_iovec(iov, iovcnt, wlimit_.avail());
-      if (iovcnt == 0) {
+      auto nwrite = conn_.write_clear(wb_.pos, wb_.rleft());
+      if (nwrite == 0) {
         return 0;
       }
-
-      ssize_t nwrite;
-      while ((nwrite = writev(fd_, iov, iovcnt)) == -1 && errno == EINTR)
-        ;
-      if (nwrite == -1) {
-        if (errno == EAGAIN || errno == EWOULDBLOCK) {
-          wlimit_.startw();
-          ev_timer_again(loop_, &wt_);
-          return 0;
-        }
+      if (nwrite < 0) {
         return -1;
       }
       wb_.drain(nwrite);
-      wlimit_.drain(nwrite);
       continue;
     }
     wb_.reset();
@@ -169,54 +154,34 @@ int ClientHandler::write_clear() {
     }
   }
 
-  wlimit_.stopw();
-  ev_timer_stop(loop_, &wt_);
+  conn_.wlimit.stopw();
+  ev_timer_stop(conn_.loop, &conn_.wt);
 
   return 0;
 }
 
 int ClientHandler::tls_handshake() {
-  ev_timer_again(loop_, &rt_);
+  ev_timer_again(conn_.loop, &conn_.rt);
 
   ERR_clear_error();
 
-  auto rv = SSL_do_handshake(ssl_);
+  auto rv = conn_.tls_handshake();
 
-  if (rv == 0) {
-    return -1;
+  if (rv == SHRPX_ERR_INPROGRESS) {
+    return 0;
   }
 
   if (rv < 0) {
-    auto err = SSL_get_error(ssl_, rv);
-    switch (err) {
-    case SSL_ERROR_WANT_READ:
-      wlimit_.stopw();
-      ev_timer_stop(loop_, &wt_);
-      return 0;
-    case SSL_ERROR_WANT_WRITE:
-      wlimit_.startw();
-      ev_timer_again(loop_, &wt_);
-      return 0;
-    default:
-      return -1;
-    }
+    return -1;
   }
 
-  wlimit_.stopw();
-  ev_timer_stop(loop_, &wt_);
-
-  set_tls_handshake(true);
   if (LOG_ENABLED(INFO)) {
     CLOG(INFO, this) << "SSL/TLS handshake completed";
   }
+
   if (validate_next_proto() != 0) {
     return -1;
   }
-  if (LOG_ENABLED(INFO)) {
-    if (SSL_session_reused(ssl_)) {
-      CLOG(INFO, this) << "SSL/TLS session reused";
-    }
-  }
 
   read_ = &ClientHandler::read_tls;
   write_ = &ClientHandler::write_tls;
@@ -225,7 +190,7 @@ int ClientHandler::tls_handshake() {
 }
 
 int ClientHandler::read_tls() {
-  ev_timer_again(loop_, &rt_);
+  ev_timer_again(conn_.loop, &conn_.rt);
 
   ERR_clear_error();
 
@@ -234,126 +199,45 @@ int ClientHandler::read_tls() {
     if (rb_.rleft() && on_read() != 0) {
       return -1;
     }
-    if (rb_.rleft()) {
+    if (rb_.rleft() == 0) {
+      rb_.reset();
+    } else if (rb_.wleft() == 0) {
+      conn_.rlimit.stopw();
       return 0;
     }
-    rb_.reset();
-    struct iovec iov[2];
-    auto iovcnt = rb_.wiovec(iov);
-    // SSL_read requires the same arguments (buf pointer and its
-    // length) on SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE.
-    // rlimit_.avail() or rlimit_.avail() may return different length
-    // than the length previously passed to SSL_read, which violates
-    // OpenSSL assumption.  To avoid this, we keep last legnth passed
-    // to SSL_read to tls_last_readlen_ if SSL_read indicated I/O
-    // blocking.
-    if (tls_last_readlen_ == 0) {
-      iovcnt = limit_iovec(iov, iovcnt, rlimit_.avail());
-      if (iovcnt == 0) {
-        return 0;
-      }
-    } else {
-      assert(iov[0].iov_len == tls_last_readlen_);
-      tls_last_readlen_ = 0;
+
+    auto nread = conn_.read_tls(rb_.last, rb_.wleft());
+
+    if (nread == 0) {
+      return 0;
     }
 
-    auto rv = SSL_read(ssl_, iov[0].iov_base, iov[0].iov_len);
-
-    if (rv == 0) {
+    if (nread < 0) {
       return -1;
     }
 
-    if (rv < 0) {
-      auto err = SSL_get_error(ssl_, rv);
-      switch (err) {
-      case SSL_ERROR_WANT_READ:
-        tls_last_readlen_ = iov[0].iov_len;
-        return 0;
-      case SSL_ERROR_WANT_WRITE:
-        if (LOG_ENABLED(INFO)) {
-          CLOG(INFO, this) << "Close connection due to TLS renegotiation";
-        }
-        return -1;
-      default:
-        if (LOG_ENABLED(INFO)) {
-          CLOG(INFO, this) << "SSL_read: SSL_get_error returned " << err;
-        }
-        return -1;
-      }
-    }
-
-    rb_.write(rv);
-    rlimit_.drain(rv);
+    rb_.write(nread);
   }
 }
 
 int ClientHandler::write_tls() {
-  ev_timer_again(loop_, &rt_);
+  ev_timer_again(conn_.loop, &conn_.rt);
 
   ERR_clear_error();
 
   for (;;) {
     if (wb_.rleft() > 0) {
-      const void *p;
-      size_t len;
-      std::tie(p, len) = wb_.get();
+      auto nwrite = conn_.write_tls(wb_.pos, wb_.rleft());
 
-      // SSL_write requires the same arguments (buf pointer and its
-      // length) on SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE.
-      // get_write_limit() may return smaller length than previously
-      // passed to SSL_write, which violates OpenSSL assumption.  To
-      // avoid this, we keep last legnth passed to SSL_write to
-      // tls_last_writelen_ if SSL_write indicated I/O blocking.
-      if (tls_last_writelen_ == 0) {
-        len = std::min(len, wlimit_.avail());
-        if (len == 0) {
-          return 0;
-        }
-
-        auto limit = get_write_limit();
-        if (limit != -1) {
-          len = std::min(len, static_cast<size_t>(limit));
-        }
-      } else {
-        assert(len >= tls_last_writelen_);
-
-        len = tls_last_writelen_;
-        tls_last_writelen_ = 0;
+      if (nwrite == 0) {
+        return 0;
       }
 
-      auto rv = SSL_write(ssl_, p, len);
-
-      if (rv == 0) {
+      if (nwrite < 0) {
         return -1;
       }
 
-      update_last_write_time();
-
-      if (rv < 0) {
-        auto err = SSL_get_error(ssl_, rv);
-        switch (err) {
-        case SSL_ERROR_WANT_READ:
-          if (LOG_ENABLED(INFO)) {
-            CLOG(INFO, this) << "Close connection due to TLS renegotiation";
-          }
-          return -1;
-        case SSL_ERROR_WANT_WRITE:
-          tls_last_writelen_ = len;
-          wlimit_.startw();
-          ev_timer_again(loop_, &wt_);
-          return 0;
-        default:
-          if (LOG_ENABLED(INFO)) {
-            CLOG(INFO, this) << "SSL_write: SSL_get_error returned " << err;
-          }
-          return -1;
-        }
-      }
-
-      wb_.drain(rv);
-      wlimit_.drain(rv);
-
-      update_warmup_writelen(rv);
+      wb_.drain(nwrite);
 
       continue;
     }
@@ -366,8 +250,8 @@ int ClientHandler::write_tls() {
     }
   }
 
-  wlimit_.stopw();
-  ev_timer_stop(loop_, &wt_);
+  conn_.wlimit.stopw();
+  ev_timer_stop(conn_.loop, &conn_.wt);
 
   return 0;
 }
@@ -396,128 +280,103 @@ int ClientHandler::upstream_write() {
 }
 
 int ClientHandler::upstream_http2_connhd_read() {
-  struct iovec iov[2];
-  auto iovcnt = rb_.riovec(iov);
-  for (int i = 0; i < iovcnt; ++i) {
-    auto nread =
-        std::min(left_connhd_len_, static_cast<size_t>(iov[i].iov_len));
-    if (memcmp(NGHTTP2_CLIENT_CONNECTION_PREFACE +
-                   NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN - left_connhd_len_,
-               iov[i].iov_base, nread) != 0) {
-      // There is no downgrade path here. Just drop the connection.
-      if (LOG_ENABLED(INFO)) {
-        CLOG(INFO, this) << "invalid client connection header";
-      }
+  auto nread = std::min(left_connhd_len_, rb_.rleft());
+  if (memcmp(NGHTTP2_CLIENT_CONNECTION_PREFACE +
+                 NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN - left_connhd_len_,
+             rb_.pos, nread) != 0) {
+    // There is no downgrade path here. Just drop the connection.
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "invalid client connection header";
+    }
 
+    return -1;
+  }
+
+  left_connhd_len_ -= nread;
+  rb_.drain(nread);
+  conn_.rlimit.startw();
+
+  if (left_connhd_len_ == 0) {
+    on_read_ = &ClientHandler::upstream_read;
+    // Run on_read to process data left in buffer since they are not
+    // notified further
+    if (on_read() != 0) {
       return -1;
     }
-
-    left_connhd_len_ -= nread;
-    rb_.drain(nread);
-
-    if (left_connhd_len_ == 0) {
-      on_read_ = &ClientHandler::upstream_read;
-      // Run on_read to process data left in buffer since they are not
-      // notified further
-      if (on_read() != 0) {
-        return -1;
-      }
-      return 0;
-    }
+    return 0;
   }
 
   return 0;
 }
 
 int ClientHandler::upstream_http1_connhd_read() {
-  struct iovec iov[2];
-  auto iovcnt = rb_.riovec(iov);
-  for (int i = 0; i < iovcnt; ++i) {
-    auto nread =
-        std::min(left_connhd_len_, static_cast<size_t>(iov[i].iov_len));
-    if (memcmp(NGHTTP2_CLIENT_CONNECTION_PREFACE +
-                   NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN - left_connhd_len_,
-               iov[i].iov_base, nread) != 0) {
-      if (LOG_ENABLED(INFO)) {
-        CLOG(INFO, this) << "This is HTTP/1.1 connection, "
-                         << "but may be upgraded to HTTP/2 later.";
-      }
-
-      // Reset header length for later HTTP/2 upgrade
-      left_connhd_len_ = NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN;
-      on_read_ = &ClientHandler::upstream_read;
-      on_write_ = &ClientHandler::upstream_write;
-
-      if (on_read() != 0) {
-        return -1;
-      }
-
-      return 0;
+  auto nread = std::min(left_connhd_len_, rb_.rleft());
+  if (memcmp(NGHTTP2_CLIENT_CONNECTION_PREFACE +
+                 NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN - left_connhd_len_,
+             rb_.pos, nread) != 0) {
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "This is HTTP/1.1 connection, "
+                       << "but may be upgraded to HTTP/2 later.";
     }
 
-    left_connhd_len_ -= nread;
-    rb_.drain(nread);
+    // Reset header length for later HTTP/2 upgrade
+    left_connhd_len_ = NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN;
+    on_read_ = &ClientHandler::upstream_read;
+    on_write_ = &ClientHandler::upstream_write;
 
-    if (left_connhd_len_ == 0) {
-      if (LOG_ENABLED(INFO)) {
-        CLOG(INFO, this) << "direct HTTP/2 connection";
-      }
-
-      direct_http2_upgrade();
-      on_read_ = &ClientHandler::upstream_read;
-      on_write_ = &ClientHandler::upstream_write;
-
-      // Run on_read to process data left in buffer since they are not
-      // notified further
-      if (on_read() != 0) {
-        return -1;
-      }
-
-      return 0;
+    if (on_read() != 0) {
+      return -1;
     }
+
+    return 0;
+  }
+
+  left_connhd_len_ -= nread;
+  rb_.drain(nread);
+  conn_.rlimit.startw();
+
+  if (left_connhd_len_ == 0) {
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "direct HTTP/2 connection";
+    }
+
+    direct_http2_upgrade();
+    on_read_ = &ClientHandler::upstream_read;
+    on_write_ = &ClientHandler::upstream_write;
+
+    // Run on_read to process data left in buffer since they are not
+    // notified further
+    if (on_read() != 0) {
+      return -1;
+    }
+
+    return 0;
   }
 
   return 0;
 }
 
-ClientHandler::ClientHandler(struct ev_loop *loop, int fd, SSL *ssl,
-                             const char *ipaddr, const char *port,
-                             WorkerStat *worker_stat,
-                             DownstreamConnectionPool *dconn_pool)
-    : ipaddr_(ipaddr), port_(port),
-      wlimit_(loop, &wev_, get_config()->write_rate, get_config()->write_burst),
-      rlimit_(loop, &rev_, get_config()->read_rate, get_config()->read_burst),
-      loop_(loop), dconn_pool_(dconn_pool), http2session_(nullptr),
-      http1_connect_blocker_(nullptr), ssl_(ssl), worker_stat_(worker_stat),
-      last_write_time_(0.), warmup_writelen_(0),
+ClientHandler::ClientHandler(Worker *worker, int fd, SSL *ssl,
+                             const char *ipaddr, const char *port)
+    : conn_(worker->get_loop(), fd, ssl, get_config()->upstream_write_timeout,
+            get_config()->upstream_read_timeout, get_config()->write_rate,
+            get_config()->write_burst, get_config()->read_rate,
+            get_config()->read_burst, writecb, readcb, timeoutcb, this),
+      ipaddr_(ipaddr), port_(port), worker_(worker),
       left_connhd_len_(NGHTTP2_CLIENT_CONNECTION_PREFACE_LEN),
-      tls_last_writelen_(0), tls_last_readlen_(0), fd_(fd),
-      should_close_after_write_(false), tls_handshake_(false),
-      tls_renegotiation_(false) {
+      should_close_after_write_(false) {
 
-  ++worker_stat->num_connections;
-
-  ev_io_init(&wev_, writecb, fd_, EV_WRITE);
-  ev_io_init(&rev_, readcb, fd_, EV_READ);
-
-  wev_.data = this;
-  rev_.data = this;
-
-  ev_timer_init(&wt_, timeoutcb, 0., get_config()->upstream_write_timeout);
-  ev_timer_init(&rt_, timeoutcb, 0., get_config()->upstream_read_timeout);
-
-  wt_.data = this;
-  rt_.data = this;
+  ++worker_->get_worker_stat()->num_connections;
 
   ev_timer_init(&reneg_shutdown_timer_, shutdowncb, 0., 0.);
 
   reneg_shutdown_timer_.data = this;
 
-  rlimit_.startw();
-  ev_timer_again(loop_, &rt_);
+  conn_.rlimit.startw();
+  ev_timer_again(conn_.loop, &conn_.rt);
 
-  if (ssl_) {
-    SSL_set_app_data(ssl_, reinterpret_cast<char *>(this));
+  if (conn_.tls.ssl) {
+    SSL_set_app_data(conn_.tls.ssl, &conn_);
     read_ = write_ = &ClientHandler::tls_handshake;
     on_read_ = &ClientHandler::upstream_noop;
     on_write_ = &ClientHandler::upstream_write;
@@ -525,7 +384,7 @@ ClientHandler::ClientHandler(struct ev_loop *loop, int fd, SSL *ssl,
     // For non-TLS version, first create HttpsUpstream. It may be
     // upgraded to HTTP/2 through HTTP Upgrade or direct HTTP/2
     // connection.
-    upstream_ = util::make_unique<HttpsUpstream>(this);
+    upstream_ = make_unique<HttpsUpstream>(this);
     alpn_ = "http/1.1";
     read_ = &ClientHandler::read_clear;
     write_ = &ClientHandler::write_clear;
@@ -543,35 +402,17 @@ ClientHandler::~ClientHandler() {
     upstream_->on_handler_delete();
   }
 
-  --worker_stat_->num_connections;
+  auto worker_stat = worker_->get_worker_stat();
+  --worker_stat->num_connections;
 
-  ev_timer_stop(loop_, &reneg_shutdown_timer_);
-
-  ev_timer_stop(loop_, &rt_);
-  ev_timer_stop(loop_, &wt_);
-
-  ev_io_stop(loop_, &rev_);
-  ev_io_stop(loop_, &wev_);
+  ev_timer_stop(conn_.loop, &reneg_shutdown_timer_);
 
   // TODO If backend is http/2, and it is in CONNECTED state, signal
   // it and make it loopbreak when output is zero.
-  if (worker_config->graceful_shutdown && worker_stat_->num_connections == 0) {
-    ev_break(loop_);
+  if (worker_->get_graceful_shutdown() && worker_stat->num_connections == 0) {
+    ev_break(conn_.loop);
   }
 
-  if (ssl_) {
-    SSL_set_app_data(ssl_, nullptr);
-    SSL_set_shutdown(ssl_, SSL_RECEIVED_SHUTDOWN);
-    ERR_clear_error();
-    SSL_shutdown(ssl_);
-  }
-
-  if (ssl_) {
-    SSL_free(ssl_);
-  }
-
-  shutdown(fd_, SHUT_WR);
-  close(fd_);
   if (LOG_ENABLED(INFO)) {
     CLOG(INFO, this) << "Deleted";
   }
@@ -580,20 +421,20 @@ ClientHandler::~ClientHandler() {
 Upstream *ClientHandler::get_upstream() { return upstream_.get(); }
 
 struct ev_loop *ClientHandler::get_loop() const {
-  return loop_;
+  return conn_.loop;
 }
 
 void ClientHandler::reset_upstream_read_timeout(ev_tstamp t) {
-  rt_.repeat = t;
-  if (ev_is_active(&rt_)) {
-    ev_timer_again(loop_, &rt_);
+  conn_.rt.repeat = t;
+  if (ev_is_active(&conn_.rt)) {
+    ev_timer_again(conn_.loop, &conn_.rt);
   }
 }
 
 void ClientHandler::reset_upstream_write_timeout(ev_tstamp t) {
-  wt_.repeat = t;
-  if (ev_is_active(&wt_)) {
-    ev_timer_again(loop_, &wt_);
+  conn_.wt.repeat = t;
+  if (ev_is_active(&conn_.wt)) {
+    ev_timer_again(conn_.loop, &conn_.wt);
   }
 }
 
@@ -605,7 +446,7 @@ int ClientHandler::validate_next_proto() {
   // First set callback for catch all cases
   on_read_ = &ClientHandler::upstream_read;
 
-  SSL_get0_next_proto_negotiated(ssl_, &next_proto, &next_proto_len);
+  SSL_get0_next_proto_negotiated(conn_.tls.ssl, &next_proto, &next_proto_len);
   for (int i = 0; i < 2; ++i) {
     if (next_proto) {
       if (LOG_ENABLED(INFO)) {
@@ -622,9 +463,9 @@ int ClientHandler::validate_next_proto() {
 
         on_read_ = &ClientHandler::upstream_http2_connhd_read;
 
-        auto http2_upstream = util::make_unique<Http2Upstream>(this);
+        auto http2_upstream = make_unique<Http2Upstream>(this);
 
-        if (!ssl::check_http2_requirement(ssl_)) {
+        if (!ssl::check_http2_requirement(conn_.tls.ssl)) {
           rv = http2_upstream->terminate_session(NGHTTP2_INADEQUATE_SECURITY);
 
           if (rv != 0) {
@@ -647,7 +488,7 @@ int ClientHandler::validate_next_proto() {
 #ifdef HAVE_SPDYLAY
         uint16_t version = spdylay_npn_get_version(next_proto, next_proto_len);
         if (version) {
-          upstream_ = util::make_unique<SpdyUpstream>(version, this);
+          upstream_ = make_unique<SpdyUpstream>(version, this);
 
           switch (version) {
           case SPDYLAY_PROTO_SPDY2:
@@ -674,7 +515,7 @@ int ClientHandler::validate_next_proto() {
         }
 #endif // HAVE_SPDYLAY
         if (next_proto_len == 8 && memcmp("http/1.1", next_proto, 8) == 0) {
-          upstream_ = util::make_unique<HttpsUpstream>(this);
+          upstream_ = make_unique<HttpsUpstream>(this);
           alpn_ = "http/1.1";
 
           // At this point, input buffer is already filled with some
@@ -690,7 +531,7 @@ int ClientHandler::validate_next_proto() {
       break;
     }
 #if OPENSSL_VERSION_NUMBER >= 0x10002000L
-    SSL_get0_alpn_selected(ssl_, &next_proto, &next_proto_len);
+    SSL_get0_alpn_selected(conn_.tls.ssl, &next_proto, &next_proto_len);
 #else  // OPENSSL_VERSION_NUMBER < 0x10002000L
     break;
 #endif // OPENSSL_VERSION_NUMBER < 0x10002000L
@@ -699,7 +540,7 @@ int ClientHandler::validate_next_proto() {
     if (LOG_ENABLED(INFO)) {
       CLOG(INFO, this) << "No protocol negotiated. Fallback to HTTP/1.1";
     }
-    upstream_ = util::make_unique<HttpsUpstream>(this);
+    upstream_ = make_unique<HttpsUpstream>(this);
     alpn_ = "http/1.1";
 
     // At this point, input buffer is already filled with some bytes.
@@ -739,7 +580,8 @@ void ClientHandler::pool_downstream_connection(
     CLOG(INFO, this) << "Pooling downstream connection DCONN:" << dconn.get();
   }
   dconn->set_client_handler(nullptr);
-  dconn_pool_->add_downstream_connection(std::move(dconn));
+  auto dconn_pool = worker_->get_dconn_pool();
+  dconn_pool->add_downstream_connection(std::move(dconn));
 }
 
 void ClientHandler::remove_downstream_connection(DownstreamConnection *dconn) {
@@ -747,12 +589,14 @@ void ClientHandler::remove_downstream_connection(DownstreamConnection *dconn) {
     CLOG(INFO, this) << "Removing downstream connection DCONN:" << dconn
                      << " from pool";
   }
-  dconn_pool_->remove_downstream_connection(dconn);
+  auto dconn_pool = worker_->get_dconn_pool();
+  dconn_pool->remove_downstream_connection(dconn);
 }
 
 std::unique_ptr<DownstreamConnection>
 ClientHandler::get_downstream_connection() {
-  auto dconn = dconn_pool_->pop_downstream_connection();
+  auto dconn_pool = worker_->get_dconn_pool();
+  auto dconn = dconn_pool->pop_downstream_connection();
 
   if (!dconn) {
     if (LOG_ENABLED(INFO)) {
@@ -760,11 +604,13 @@ ClientHandler::get_downstream_connection() {
                        << " Create new one";
     }
 
-    if (http2session_) {
-      dconn = util::make_unique<Http2DownstreamConnection>(dconn_pool_,
-                                                           http2session_);
+    auto dconn_pool = worker_->get_dconn_pool();
+    auto http2session = worker_->get_http2_session();
+
+    if (http2session) {
+      dconn = make_unique<Http2DownstreamConnection>(dconn_pool, http2session);
     } else {
-      dconn = util::make_unique<HttpDownstreamConnection>(dconn_pool_, loop_);
+      dconn = make_unique<HttpDownstreamConnection>(dconn_pool, conn_.loop);
     }
     dconn->set_client_handler(this);
     return dconn;
@@ -780,25 +626,18 @@ ClientHandler::get_downstream_connection() {
   return dconn;
 }
 
-SSL *ClientHandler::get_ssl() const { return ssl_; }
+SSL *ClientHandler::get_ssl() const { return conn_.tls.ssl; }
 
-void ClientHandler::set_http2_session(Http2Session *http2session) {
-  http2session_ = http2session;
-}
-
-Http2Session *ClientHandler::get_http2_session() const { return http2session_; }
-
-void ClientHandler::set_http1_connect_blocker(
-    ConnectBlocker *http1_connect_blocker) {
-  http1_connect_blocker_ = http1_connect_blocker;
+Http2Session *ClientHandler::get_http2_session() const {
+  return worker_->get_http2_session();
 }
 
 ConnectBlocker *ClientHandler::get_http1_connect_blocker() const {
-  return http1_connect_blocker_;
+  return worker_->get_http1_connect_blocker();
 }
 
 void ClientHandler::direct_http2_upgrade() {
-  upstream_ = util::make_unique<Http2Upstream>(this);
+  upstream_ = make_unique<Http2Upstream>(this);
   // TODO We don't know exact h2 draft version in direct upgrade.  We
   // just use library default for now.
   alpn_ = NGHTTP2_CLEARTEXT_PROTO_VERSION_ID;
@@ -806,7 +645,7 @@ void ClientHandler::direct_http2_upgrade() {
 }
 
 int ClientHandler::perform_http2_upgrade(HttpsUpstream *http) {
-  auto upstream = util::make_unique<Http2Upstream>(this);
+  auto upstream = make_unique<Http2Upstream>(this);
   if (upstream->upgrade_upstream(http) != 0) {
     return -1;
   }
@@ -826,70 +665,18 @@ int ClientHandler::perform_http2_upgrade(HttpsUpstream *http) {
   return 0;
 }
 
-bool ClientHandler::get_http2_upgrade_allowed() const { return !ssl_; }
+bool ClientHandler::get_http2_upgrade_allowed() const { return !conn_.tls.ssl; }
 
 std::string ClientHandler::get_upstream_scheme() const {
-  if (ssl_) {
+  if (conn_.tls.ssl) {
     return "https";
   } else {
     return "http";
   }
 }
 
-void ClientHandler::set_tls_handshake(bool f) { tls_handshake_ = f; }
-
-bool ClientHandler::get_tls_handshake() const { return tls_handshake_; }
-
-void ClientHandler::set_tls_renegotiation(bool f) {
-  if (tls_renegotiation_ == false) {
-    if (LOG_ENABLED(INFO)) {
-      CLOG(INFO, this) << "TLS renegotiation detected. "
-                       << "Start shutdown timer now.";
-    }
-
-    ev_timer_start(loop_, &reneg_shutdown_timer_);
-  }
-  tls_renegotiation_ = f;
-}
-
-bool ClientHandler::get_tls_renegotiation() const { return tls_renegotiation_; }
-
-namespace {
-const size_t SHRPX_SMALL_WRITE_LIMIT = 1300;
-const size_t SHRPX_WARMUP_THRESHOLD = 1 << 20;
-} // namespace
-
-ssize_t ClientHandler::get_write_limit() {
-  if (!ssl_) {
-    return -1;
-  }
-
-  auto t = ev_now(loop_);
-
-  if (t - last_write_time_ > 1.0) {
-    // Time out, use small record size
-    warmup_writelen_ = 0;
-    return SHRPX_SMALL_WRITE_LIMIT;
-  }
-
-  // If event_base_gettimeofday_cached() failed, we just skip timer
-  // checking.  Don't know how to treat this.
-
-  if (warmup_writelen_ >= SHRPX_WARMUP_THRESHOLD) {
-    return -1;
-  }
-
-  return SHRPX_SMALL_WRITE_LIMIT;
-}
-
-void ClientHandler::update_warmup_writelen(size_t n) {
-  if (warmup_writelen_ < SHRPX_WARMUP_THRESHOLD) {
-    warmup_writelen_ += n;
-  }
-}
-
-void ClientHandler::update_last_write_time() {
-  last_write_time_ = ev_now(loop_);
+void ClientHandler::start_immediate_shutdown() {
+  ev_timer_start(conn_.loop, &reneg_shutdown_timer_);
 }
 
 void ClientHandler::write_accesslog(Downstream *downstream) {
@@ -936,15 +723,17 @@ void ClientHandler::write_accesslog(int major, int minor, unsigned int status,
   upstream_accesslog(get_config()->accesslog_format, &lgsp);
 }
 
-WorkerStat *ClientHandler::get_worker_stat() const { return worker_stat_; }
-
 ClientHandler::WriteBuf *ClientHandler::get_wb() { return &wb_; }
 
 ClientHandler::ReadBuf *ClientHandler::get_rb() { return &rb_; }
 
-void ClientHandler::signal_write() { wlimit_.startw(); }
+void ClientHandler::signal_write() { conn_.wlimit.startw(); }
 
-RateLimit *ClientHandler::get_rlimit() { return &rlimit_; }
-RateLimit *ClientHandler::get_wlimit() { return &wlimit_; }
+RateLimit *ClientHandler::get_rlimit() { return &conn_.rlimit; }
+RateLimit *ClientHandler::get_wlimit() { return &conn_.wlimit; }
+
+ev_io *ClientHandler::get_wev() { return &conn_.wev; }
+
+Worker *ClientHandler::get_worker() const { return worker_; }
 
 } // namespace shrpx

src/shrpx_client_handler.h

@@ -34,7 +34,8 @@
 #include <openssl/ssl.h>
 
 #include "shrpx_rate_limit.h"
-#include "ringbuf.h"
+#include "shrpx_connection.h"
+#include "buffer.h"
 
 using namespace nghttp2;
 
@@ -46,13 +47,13 @@ class Http2Session;
 class HttpsUpstream;
 class ConnectBlocker;
 class DownstreamConnectionPool;
+class Worker;
 struct WorkerStat;
 
 class ClientHandler {
 public:
-  ClientHandler(struct ev_loop *loop, int fd, SSL *ssl, const char *ipaddr,
-                const char *port, WorkerStat *worker_stat,
-                DownstreamConnectionPool *dconn_pool);
+  ClientHandler(Worker *worker, int fd, SSL *ssl, const char *ipaddr,
+                const char *port);
   ~ClientHandler();
 
   // Performs clear text I/O
@@ -92,9 +93,7 @@ public:
   void remove_downstream_connection(DownstreamConnection *dconn);
   std::unique_ptr<DownstreamConnection> get_downstream_connection();
   SSL *get_ssl() const;
-  void set_http2_session(Http2Session *http2session);
   Http2Session *get_http2_session() const;
-  void set_http1_connect_blocker(ConnectBlocker *http1_connect_blocker);
   ConnectBlocker *get_http1_connect_blocker() const;
   // Call this function when HTTP/2 connection header is received at
   // the start of the connection.
@@ -106,24 +105,7 @@ public:
   bool get_http2_upgrade_allowed() const;
   // Returns upstream scheme, either "http" or "https"
   std::string get_upstream_scheme() const;
-  void set_tls_handshake(bool f);
-  bool get_tls_handshake() const;
-  void set_tls_renegotiation(bool f);
-  bool get_tls_renegotiation() const;
-  // Returns maximum chunk size for one evbuffer_add().  The intention
-  // of this chunk size is control the TLS record size.  The actual
-  // SSL_write() call is done under libevent control.  In
-  // libevent-2.0.21, libevent calls SSL_write() for each chunk inside
-  // evbuffer.  This means that we can control TLS record size by
-  // adjusting the chunk size to evbuffer_add().
-  //
-  // This function returns -1, if TLS is not enabled or no limitation
-  // is required.
-  ssize_t get_write_limit();
-  // Updates the number of bytes written in warm up period.
-  void update_warmup_writelen(size_t n);
-  // Updates the time when last write was done.
-  void update_last_write_time();
+  void start_immediate_shutdown();
 
   // Writes upstream accesslog using |downstream|.  The |downstream|
   // must not be nullptr.
@@ -133,10 +115,10 @@ public:
   // corresponding Downstream object is not available.
   void write_accesslog(int major, int minor, unsigned int status,
                        int64_t body_bytes_sent);
-  WorkerStat *get_worker_stat() const;
+  Worker *get_worker() const;
 
-  using WriteBuf = RingBuf<65536>;
-  using ReadBuf = RingBuf<8192>;
+  using WriteBuf = Buffer<32768>;
+  using ReadBuf = Buffer<8192>;
 
   WriteBuf *get_wb();
   ReadBuf *get_rb();
@@ -145,12 +127,10 @@ public:
   RateLimit *get_wlimit();
 
   void signal_write();
+  ev_io *get_wev();
 
 private:
-  ev_io wev_;
-  ev_io rev_;
-  ev_timer wt_;
-  ev_timer rt_;
+  Connection conn_;
   ev_timer reneg_shutdown_timer_;
   std::unique_ptr<Upstream> upstream_;
   std::string ipaddr_;
@@ -159,26 +139,10 @@ private:
   std::string alpn_;
   std::function<int(ClientHandler &)> read_, write_;
   std::function<int(ClientHandler &)> on_read_, on_write_;
-  RateLimit wlimit_;
-  RateLimit rlimit_;
-  struct ev_loop *loop_;
-  DownstreamConnectionPool *dconn_pool_;
-  // Shared HTTP2 session for each thread. NULL if backend is not
-  // HTTP2. Not deleted by this object.
-  Http2Session *http2session_;
-  ConnectBlocker *http1_connect_blocker_;
-  SSL *ssl_;
-  WorkerStat *worker_stat_;
-  double last_write_time_;
-  size_t warmup_writelen_;
+  Worker *worker_;
   // The number of bytes of HTTP/2 client connection header to read
   size_t left_connhd_len_;
-  size_t tls_last_writelen_;
-  size_t tls_last_readlen_;
-  int fd_;
   bool should_close_after_write_;
-  bool tls_handshake_;
-  bool tls_renegotiation_;
   WriteBuf wb_;
   ReadBuf rb_;
 };

src/shrpx_config.cc

@@ -46,6 +46,7 @@
 #include "shrpx_http.h"
 #include "http2.h"
 #include "util.h"
+#include "template.h"
 
 using namespace nghttp2;
 
@@ -133,6 +134,7 @@ const char SHRPX_OPT_ADD_RESPONSE_HEADER[] = "add-response-header";
 const char SHRPX_OPT_WORKER_FRONTEND_CONNECTIONS[] =
     "worker-frontend-connections";
 const char SHRPX_OPT_NO_LOCATION_REWRITE[] = "no-location-rewrite";
+const char SHRPX_OPT_NO_HOST_REWRITE[] = "no-host-rewrite";
 const char SHRPX_OPT_BACKEND_HTTP1_CONNECTIONS_PER_HOST[] =
     "backend-http1-connections-per-host";
 const char SHRPX_OPT_BACKEND_HTTP1_CONNECTIONS_PER_FRONTEND[] =
@@ -143,6 +145,7 @@ const char SHRPX_OPT_RLIMIT_NOFILE[] = "rlimit-nofile";
 const char SHRPX_OPT_TLS_CTX_PER_WORKER[] = "tls-ctx-per-worker";
 const char SHRPX_OPT_BACKEND_REQUEST_BUFFER[] = "backend-request-buffer";
 const char SHRPX_OPT_BACKEND_RESPONSE_BUFFER[] = "backend-response-buffer";
+const char SHRPX_OPT_NO_SERVER_PUSH[] = "no-server-push";
 
 namespace {
 Config *config = nullptr;
@@ -207,7 +210,7 @@ bool is_secure(const char *filename) {
 
 std::unique_ptr<TicketKeys>
 read_tls_ticket_key_file(const std::vector<std::string> &files) {
-  auto ticket_keys = util::make_unique<TicketKeys>();
+  auto ticket_keys = make_unique<TicketKeys>();
   auto &keys = ticket_keys->keys;
   keys.resize(files.size());
   size_t i = 0;
@@ -250,7 +253,7 @@ FILE *open_file_for_write(const char *filename) {
 
   // We get race condition if execve is called at the same time.
   if (fd != -1) {
-    util::make_socket_closeonexec(fd);
+    make_socket_closeonexec(fd);
   }
 #endif
   if (fd == -1) {
@@ -292,7 +295,7 @@ std::unique_ptr<char[]> strcopy(const char *val) {
 }
 
 std::unique_ptr<char[]> strcopy(const char *val, size_t len) {
-  auto res = util::make_unique<char[]>(len + 1);
+  auto res = make_unique<char[]>(len + 1);
   memcpy(res.get(), val, len);
   res[len] = '\0';
   return res;
@@ -439,31 +442,31 @@ std::vector<LogFragment> parse_log_format(const char *optarg) {
     const char *value = nullptr;
     size_t valuelen = 0;
 
-    if (util::strieq("$remote_addr", var_start, varlen)) {
+    if (util::strieq_l("$remote_addr", var_start, varlen)) {
       type = SHRPX_LOGF_REMOTE_ADDR;
-    } else if (util::strieq("$time_local", var_start, varlen)) {
+    } else if (util::strieq_l("$time_local", var_start, varlen)) {
       type = SHRPX_LOGF_TIME_LOCAL;
-    } else if (util::strieq("$time_iso8601", var_start, varlen)) {
+    } else if (util::strieq_l("$time_iso8601", var_start, varlen)) {
       type = SHRPX_LOGF_TIME_ISO8601;
-    } else if (util::strieq("$request", var_start, varlen)) {
+    } else if (util::strieq_l("$request", var_start, varlen)) {
       type = SHRPX_LOGF_REQUEST;
-    } else if (util::strieq("$status", var_start, varlen)) {
+    } else if (util::strieq_l("$status", var_start, varlen)) {
       type = SHRPX_LOGF_STATUS;
-    } else if (util::strieq("$body_bytes_sent", var_start, varlen)) {
+    } else if (util::strieq_l("$body_bytes_sent", var_start, varlen)) {
       type = SHRPX_LOGF_BODY_BYTES_SENT;
     } else if (util::istartsWith(var_start, varlen, "$http_")) {
       type = SHRPX_LOGF_HTTP;
       value = var_start + sizeof("$http_") - 1;
       valuelen = varlen - (sizeof("$http_") - 1);
-    } else if (util::strieq("$remote_port", var_start, varlen)) {
+    } else if (util::strieq_l("$remote_port", var_start, varlen)) {
       type = SHRPX_LOGF_REMOTE_PORT;
-    } else if (util::strieq("$server_port", var_start, varlen)) {
+    } else if (util::strieq_l("$server_port", var_start, varlen)) {
       type = SHRPX_LOGF_SERVER_PORT;
-    } else if (util::strieq("$request_time", var_start, varlen)) {
+    } else if (util::strieq_l("$request_time", var_start, varlen)) {
       type = SHRPX_LOGF_REQUEST_TIME;
-    } else if (util::strieq("$pid", var_start, varlen)) {
+    } else if (util::strieq_l("$pid", var_start, varlen)) {
       type = SHRPX_LOGF_PID;
-    } else if (util::strieq("$alpn", var_start, varlen)) {
+    } else if (util::strieq_l("$alpn", var_start, varlen)) {
       type = SHRPX_LOGF_ALPN;
     } else {
       LOG(WARN) << "Unrecognized log format variable: "
@@ -501,14 +504,14 @@ std::vector<LogFragment> parse_log_format(const char *optarg) {
 }
 
 namespace {
-int parse_timeval(ev_tstamp *dest, const char *opt, const char *optarg) {
-  time_t sec;
-
-  if (parse_uint(&sec, opt, optarg) != 0) {
+int parse_duration(ev_tstamp *dest, const char *opt, const char *optarg) {
+  auto t = util::parse_duration_with_unit(optarg);
+  if (t == std::numeric_limits<double>::infinity()) {
+    LOG(ERROR) << opt << ": bad value: '" << optarg << "'";
     return -1;
   }
 
-  *dest = sec;
+  *dest = t;
 
   return 0;
 }
@@ -518,6 +521,17 @@ int parse_config(const char *opt, const char *optarg) {
   char host[NI_MAXHOST];
   uint16_t port;
   if (util::strieq(opt, SHRPX_OPT_BACKEND)) {
+    if (util::istartsWith(optarg, SHRPX_UNIX_PATH_PREFIX)) {
+      DownstreamAddr addr;
+      auto path = optarg + str_size(SHRPX_UNIX_PATH_PREFIX);
+      addr.host = strcopy(path);
+      addr.host_unix = true;
+
+      mod_config()->downstream_addrs.push_back(std::move(addr));
+
+      return 0;
+    }
+
     if (split_host_port(host, sizeof(host), &port, optarg) == -1) {
       return -1;
     }
@@ -532,12 +546,22 @@ int parse_config(const char *opt, const char *optarg) {
   }
 
   if (util::strieq(opt, SHRPX_OPT_FRONTEND)) {
+    if (util::istartsWith(optarg, SHRPX_UNIX_PATH_PREFIX)) {
+      auto path = optarg + str_size(SHRPX_UNIX_PATH_PREFIX);
+      mod_config()->host = strcopy(path);
+      mod_config()->port = 0;
+      mod_config()->host_unix = true;
+
+      return 0;
+    }
+
     if (split_host_port(host, sizeof(host), &port, optarg) == -1) {
       return -1;
     }
 
     mod_config()->host = strcopy(host);
     mod_config()->port = port;
+    mod_config()->host_unix = false;
 
     return 0;
   }
@@ -602,32 +626,32 @@ int parse_config(const char *opt, const char *optarg) {
   }
 
   if (util::strieq(opt, SHRPX_OPT_FRONTEND_HTTP2_READ_TIMEOUT)) {
-    return parse_timeval(&mod_config()->http2_upstream_read_timeout, opt,
-                         optarg);
+    return parse_duration(&mod_config()->http2_upstream_read_timeout, opt,
+                          optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_FRONTEND_READ_TIMEOUT)) {
-    return parse_timeval(&mod_config()->upstream_read_timeout, opt, optarg);
+    return parse_duration(&mod_config()->upstream_read_timeout, opt, optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_FRONTEND_WRITE_TIMEOUT)) {
-    return parse_timeval(&mod_config()->upstream_write_timeout, opt, optarg);
+    return parse_duration(&mod_config()->upstream_write_timeout, opt, optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_BACKEND_READ_TIMEOUT)) {
-    return parse_timeval(&mod_config()->downstream_read_timeout, opt, optarg);
+    return parse_duration(&mod_config()->downstream_read_timeout, opt, optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_BACKEND_WRITE_TIMEOUT)) {
-    return parse_timeval(&mod_config()->downstream_write_timeout, opt, optarg);
+    return parse_duration(&mod_config()->downstream_write_timeout, opt, optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_STREAM_READ_TIMEOUT)) {
-    return parse_timeval(&mod_config()->stream_read_timeout, opt, optarg);
+    return parse_duration(&mod_config()->stream_read_timeout, opt, optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_STREAM_WRITE_TIMEOUT)) {
-    return parse_timeval(&mod_config()->stream_write_timeout, opt, optarg);
+    return parse_duration(&mod_config()->stream_write_timeout, opt, optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_ACCESSLOG_FILE)) {
@@ -661,8 +685,8 @@ int parse_config(const char *opt, const char *optarg) {
   }
 
   if (util::strieq(opt, SHRPX_OPT_BACKEND_KEEP_ALIVE_TIMEOUT)) {
-    return parse_timeval(&mod_config()->downstream_idle_read_timeout, opt,
-                         optarg);
+    return parse_duration(&mod_config()->downstream_idle_read_timeout, opt,
+                          optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_FRONTEND_HTTP2_WINDOW_BITS) ||
@@ -756,6 +780,7 @@ int parse_config(const char *opt, const char *optarg) {
                  << strerror(errno);
       return -1;
     }
+    mod_config()->user = strcopy(pwd->pw_name);
     mod_config()->uid = pwd->pw_uid;
     mod_config()->gid = pwd->pw_gid;
 
@@ -1078,6 +1103,12 @@ int parse_config(const char *opt, const char *optarg) {
     return 0;
   }
 
+  if (util::strieq(opt, SHRPX_OPT_NO_HOST_REWRITE)) {
+    mod_config()->no_host_rewrite = util::strieq(optarg, "yes");
+
+    return 0;
+  }
+
   if (util::strieq(opt, SHRPX_OPT_BACKEND_HTTP1_CONNECTIONS_PER_HOST)) {
     int n;
 
@@ -1102,7 +1133,7 @@ int parse_config(const char *opt, const char *optarg) {
   }
 
   if (util::strieq(opt, SHRPX_OPT_LISTENER_DISABLE_TIMEOUT)) {
-    return parse_timeval(&mod_config()->listener_disable_timeout, opt, optarg);
+    return parse_duration(&mod_config()->listener_disable_timeout, opt, optarg);
   }
 
   if (util::strieq(opt, SHRPX_OPT_TLS_TICKET_KEY_FILE)) {
@@ -1156,6 +1187,12 @@ int parse_config(const char *opt, const char *optarg) {
     return 0;
   }
 
+  if (util::strieq(opt, SHRPX_OPT_NO_SERVER_PUSH)) {
+    mod_config()->no_server_push = util::strieq(optarg, "yes");
+
+    return 0;
+  }
+
   if (util::strieq(opt, "conf")) {
     LOG(WARN) << "conf: ignored";
 

src/shrpx_config.h

@@ -30,6 +30,7 @@
 #include <stdint.h>
 #include <sys/types.h>
 #include <sys/socket.h>
+#include <sys/un.h>
 #include <netinet/in.h>
 #include <arpa/inet.h>
 #include <cstdio>
@@ -48,10 +49,12 @@ struct LogFragment;
 
 namespace ssl {
 
-struct CertLookupTree;
+class CertLookupTree;
 
 } // namespace ssl
 
+#define SHRPX_UNIX_PATH_PREFIX "unix:"
+
 extern const char SHRPX_OPT_PRIVATE_KEY_FILE[];
 extern const char SHRPX_OPT_PRIVATE_KEY_PASSWD_FILE[];
 extern const char SHRPX_OPT_CERTIFICATE_FILE[];
@@ -123,6 +126,7 @@ extern const char SHRPX_OPT_ALTSVC[];
 extern const char SHRPX_OPT_ADD_RESPONSE_HEADER[];
 extern const char SHRPX_OPT_WORKER_FRONTEND_CONNECTIONS[];
 extern const char SHRPX_OPT_NO_LOCATION_REWRITE[];
+extern const char SHRPX_OPT_NO_HOST_REWRITE[];
 extern const char SHRPX_OPT_BACKEND_HTTP1_CONNECTIONS_PER_HOST[];
 extern const char SHRPX_OPT_BACKEND_HTTP1_CONNECTIONS_PER_FRONTEND[];
 extern const char SHRPX_OPT_LISTENER_DISABLE_TIMEOUT[];
@@ -131,12 +135,14 @@ extern const char SHRPX_OPT_RLIMIT_NOFILE[];
 extern const char SHRPX_OPT_TLS_CTX_PER_WORKER[];
 extern const char SHRPX_OPT_BACKEND_REQUEST_BUFFER[];
 extern const char SHRPX_OPT_BACKEND_RESPONSE_BUFFER[];
+extern const char SHRPX_OPT_NO_SERVER_PUSH[];
 
 union sockaddr_union {
   sockaddr_storage storage;
   sockaddr sa;
   sockaddr_in6 in6;
   sockaddr_in in;
+  sockaddr_un un;
 };
 
 enum shrpx_proto { PROTO_HTTP2, PROTO_HTTP };
@@ -158,12 +164,17 @@ struct AltSvc {
 };
 
 struct DownstreamAddr {
-  DownstreamAddr() : addr{{0}}, addrlen(0), port(0) {}
+  DownstreamAddr() : addr{{0}}, addrlen(0), port(0), host_unix(false) {}
   sockaddr_union addr;
+  // backend address.  If |host_unix| is true, this is UNIX domain
+  // socket path.
   std::unique_ptr<char[]> host;
   std::unique_ptr<char[]> hostport;
   size_t addrlen;
+  // backend port.  0 if |host_unix| is true.
   uint16_t port;
+  // true if |host| contains UNIX domain socket path.
+  bool host_unix;
 };
 
 struct TicketKey {
@@ -197,6 +208,8 @@ struct Config {
   ev_tstamp stream_write_timeout;
   ev_tstamp downstream_idle_read_timeout;
   ev_tstamp listener_disable_timeout;
+  // address of frontend connection.  This could be a path to UNIX
+  // domain socket.  In this case, |host_unix| must be true.
   std::unique_ptr<char[]> host;
   std::unique_ptr<char[]> private_key_file;
   std::unique_ptr<char[]> private_key_passwd;
@@ -234,7 +247,10 @@ struct Config {
   std::unique_ptr<char[]> errorlog_file;
   FILE *http2_upstream_dump_request_header;
   FILE *http2_upstream_dump_response_header;
+  nghttp2_session_callbacks *http2_upstream_callbacks;
+  nghttp2_session_callbacks *http2_downstream_callbacks;
   nghttp2_option *http2_option;
+  nghttp2_option *http2_client_option;
   char **argv;
   char *cwd;
   size_t num_worker;
@@ -268,9 +284,12 @@ struct Config {
   int syslog_facility;
   int backlog;
   int argc;
+  std::unique_ptr<char[]> user;
   uid_t uid;
   gid_t gid;
   pid_t pid;
+  // frontend listening port.  0 if frontend listens on UNIX domain
+  // socket, in this case |host_unix| must be true.
   uint16_t port;
   // port in http proxy URI
   uint16_t downstream_http_proxy_port;
@@ -298,8 +317,11 @@ struct Config {
   bool http2_no_cookie_crumbling;
   bool upstream_frame_debug;
   bool no_location_rewrite;
-  bool auto_tls_ticket_key;
+  bool no_host_rewrite;
   bool tls_ctx_per_worker;
+  bool no_server_push;
+  // true if host contains UNIX domain socket path
+  bool host_unix;
 };
 
 const Config *get_config();

src/shrpx_connection.cc

@@ -0,0 +1,324 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2015 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "shrpx_connection.h"
+
+#include <unistd.h>
+
+#include <limits>
+
+#include <openssl/err.h>
+
+#include "memchunk.h"
+
+using namespace nghttp2;
+
+namespace shrpx {
+Connection::Connection(struct ev_loop *loop, int fd, SSL *ssl,
+                       ev_tstamp write_timeout, ev_tstamp read_timeout,
+                       size_t write_rate, size_t write_burst, size_t read_rate,
+                       size_t read_burst, IOCb writecb, IOCb readcb,
+                       TimerCb timeoutcb, void *data)
+    : tls{ssl}, wlimit(loop, &wev, write_rate, write_burst),
+      rlimit(loop, &rev, read_rate, read_burst), writecb(writecb),
+      readcb(readcb), timeoutcb(timeoutcb), loop(loop), data(data), fd(fd) {
+
+  ev_io_init(&wev, writecb, fd, EV_WRITE);
+  ev_io_init(&rev, readcb, fd, EV_READ);
+
+  wev.data = this;
+  rev.data = this;
+
+  ev_timer_init(&wt, timeoutcb, 0., write_timeout);
+  ev_timer_init(&rt, timeoutcb, 0., read_timeout);
+
+  wt.data = this;
+  rt.data = this;
+
+  // set 0. to double field explicitly just in case
+  tls.last_write_time = 0.;
+}
+
+Connection::~Connection() { disconnect(); }
+
+void Connection::disconnect() {
+  ev_timer_stop(loop, &rt);
+  ev_timer_stop(loop, &wt);
+
+  rlimit.stopw();
+  wlimit.stopw();
+
+  if (tls.ssl) {
+    SSL_set_app_data(tls.ssl, nullptr);
+    SSL_set_shutdown(tls.ssl, SSL_RECEIVED_SHUTDOWN);
+    ERR_clear_error();
+    SSL_shutdown(tls.ssl);
+    SSL_free(tls.ssl);
+    tls.ssl = nullptr;
+  }
+
+  if (fd != -1) {
+    shutdown(fd, SHUT_WR);
+    close(fd);
+    fd = -1;
+  }
+}
+
+int Connection::tls_handshake() {
+  auto rv = SSL_do_handshake(tls.ssl);
+
+  if (rv == 0) {
+    return SHRPX_ERR_NETWORK;
+  }
+
+  if (rv < 0) {
+    auto err = SSL_get_error(tls.ssl, rv);
+    switch (err) {
+    case SSL_ERROR_WANT_READ:
+      wlimit.stopw();
+      ev_timer_stop(loop, &wt);
+      return SHRPX_ERR_INPROGRESS;
+    case SSL_ERROR_WANT_WRITE:
+      wlimit.startw();
+      ev_timer_again(loop, &wt);
+      return SHRPX_ERR_INPROGRESS;
+    default:
+      return SHRPX_ERR_NETWORK;
+    }
+  }
+
+  wlimit.stopw();
+  ev_timer_stop(loop, &wt);
+
+  tls.initial_handshake_done = true;
+
+  if (LOG_ENABLED(INFO)) {
+    LOG(INFO) << "SSL/TLS handshake completed";
+    if (SSL_session_reused(tls.ssl)) {
+      LOG(INFO) << "SSL/TLS session reused";
+    }
+  }
+
+  return 0;
+}
+
+namespace {
+const size_t SHRPX_SMALL_WRITE_LIMIT = 1300;
+const size_t SHRPX_WARMUP_THRESHOLD = 1 << 20;
+} // namespace
+
+size_t Connection::get_tls_write_limit() {
+  auto t = ev_now(loop);
+
+  if (t - tls.last_write_time > 1.) {
+    // Time out, use small record size
+    tls.warmup_writelen = 0;
+    return SHRPX_SMALL_WRITE_LIMIT;
+  }
+
+  if (tls.warmup_writelen >= SHRPX_WARMUP_THRESHOLD) {
+    return std::numeric_limits<ssize_t>::max();
+  }
+
+  return SHRPX_SMALL_WRITE_LIMIT;
+}
+
+void Connection::update_tls_warmup_writelen(size_t n) {
+  if (tls.warmup_writelen < SHRPX_WARMUP_THRESHOLD) {
+    tls.warmup_writelen += n;
+  }
+}
+
+ssize_t Connection::write_tls(const void *data, size_t len) {
+  // SSL_write requires the same arguments (buf pointer and its
+  // length) on SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE.
+  // get_write_limit() may return smaller length than previously
+  // passed to SSL_write, which violates OpenSSL assumption.  To avoid
+  // this, we keep last legnth passed to SSL_write to
+  // tls.last_writelen if SSL_write indicated I/O blocking.
+  if (tls.last_writelen == 0) {
+    len = std::min(len, wlimit.avail());
+    len = std::min(len, get_tls_write_limit());
+    if (len == 0) {
+      return 0;
+    }
+  } else {
+    len = tls.last_writelen;
+    tls.last_writelen = 0;
+  }
+
+  auto rv = SSL_write(tls.ssl, data, len);
+
+  if (rv == 0) {
+    return SHRPX_ERR_NETWORK;
+  }
+
+  tls.last_write_time = ev_now(loop);
+
+  if (rv < 0) {
+    auto err = SSL_get_error(tls.ssl, rv);
+    switch (err) {
+    case SSL_ERROR_WANT_READ:
+      if (LOG_ENABLED(INFO)) {
+        LOG(INFO) << "Close connection due to TLS renegotiation";
+      }
+      return SHRPX_ERR_NETWORK;
+    case SSL_ERROR_WANT_WRITE:
+      tls.last_writelen = len;
+      wlimit.startw();
+      ev_timer_again(loop, &wt);
+      return 0;
+    default:
+      if (LOG_ENABLED(INFO)) {
+        LOG(INFO) << "SSL_write: SSL_get_error returned " << err;
+      }
+      return SHRPX_ERR_NETWORK;
+    }
+  }
+
+  wlimit.drain(rv);
+
+  update_tls_warmup_writelen(rv);
+
+  return rv;
+}
+
+ssize_t Connection::read_tls(void *data, size_t len) {
+  // SSL_read requires the same arguments (buf pointer and its
+  // length) on SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE.
+  // rlimit_.avail() or rlimit_.avail() may return different length
+  // than the length previously passed to SSL_read, which violates
+  // OpenSSL assumption.  To avoid this, we keep last legnth passed
+  // to SSL_read to tls_last_readlen_ if SSL_read indicated I/O
+  // blocking.
+  if (tls.last_readlen == 0) {
+    len = std::min(len, rlimit.avail());
+    if (len == 0) {
+      return 0;
+    }
+  } else {
+    len = tls.last_readlen;
+    tls.last_readlen = 0;
+  }
+
+  auto rv = SSL_read(tls.ssl, data, len);
+
+  if (rv <= 0) {
+    auto err = SSL_get_error(tls.ssl, rv);
+    switch (err) {
+    case SSL_ERROR_WANT_READ:
+      tls.last_readlen = len;
+      return 0;
+    case SSL_ERROR_WANT_WRITE:
+      if (LOG_ENABLED(INFO)) {
+        LOG(INFO) << "Close connection due to TLS renegotiation";
+      }
+      return SHRPX_ERR_NETWORK;
+    case SSL_ERROR_ZERO_RETURN:
+      return SHRPX_ERR_EOF;
+    default:
+      if (LOG_ENABLED(INFO)) {
+        LOG(INFO) << "SSL_read: SSL_get_error returned " << err;
+      }
+      return SHRPX_ERR_NETWORK;
+    }
+  }
+
+  rlimit.drain(rv);
+
+  return rv;
+}
+
+ssize_t Connection::write_clear(const void *data, size_t len) {
+  len = std::min(len, wlimit.avail());
+  if (len == 0) {
+    return 0;
+  }
+
+  ssize_t nwrite;
+  while ((nwrite = write(fd, data, len)) == -1 && errno == EINTR)
+    ;
+  if (nwrite == -1) {
+    if (errno == EAGAIN || errno == EWOULDBLOCK) {
+      wlimit.startw();
+      ev_timer_again(loop, &wt);
+      return 0;
+    }
+    return SHRPX_ERR_NETWORK;
+  }
+
+  wlimit.drain(nwrite);
+
+  return nwrite;
+}
+
+ssize_t Connection::writev_clear(struct iovec *iov, int iovcnt) {
+  iovcnt = limit_iovec(iov, iovcnt, wlimit.avail());
+  if (iovcnt == 0) {
+    return 0;
+  }
+
+  ssize_t nwrite;
+  while ((nwrite = writev(fd, iov, iovcnt)) == -1 && errno == EINTR)
+    ;
+  if (nwrite == -1) {
+    if (errno == EAGAIN || errno == EWOULDBLOCK) {
+      wlimit.startw();
+      ev_timer_again(loop, &wt);
+      return 0;
+    }
+    return SHRPX_ERR_NETWORK;
+  }
+
+  wlimit.drain(nwrite);
+
+  return nwrite;
+}
+
+ssize_t Connection::read_clear(void *data, size_t len) {
+  len = std::min(len, rlimit.avail());
+  if (len == 0) {
+    return 0;
+  }
+
+  ssize_t nread;
+  while ((nread = read(fd, data, len)) == -1 && errno == EINTR)
+    ;
+  if (nread == -1) {
+    if (errno == EAGAIN || errno == EWOULDBLOCK) {
+      return 0;
+    }
+    return SHRPX_ERR_NETWORK;
+  }
+
+  if (nread == 0) {
+    return SHRPX_ERR_EOF;
+  }
+
+  rlimit.drain(nread);
+
+  return nread;
+}
+
+} // namespace shrpx

src/shrpx_connection.h

@@ -0,0 +1,107 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2015 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef SHRPX_CONNECTION_H
+#define SHRPX_CONNECTION_H
+
+#include "shrpx_config.h"
+
+#include <sys/uio.h>
+
+#include <ev.h>
+
+#include <openssl/ssl.h>
+
+#include "shrpx_rate_limit.h"
+#include "shrpx_error.h"
+
+namespace shrpx {
+
+struct TLSConnection {
+  SSL *ssl;
+  ev_tstamp last_write_time;
+  size_t warmup_writelen;
+  // length passed to SSL_write and SSL_read last time.  This is
+  // required since these functions require the exact same parameters
+  // on non-blocking I/O.
+  size_t last_writelen, last_readlen;
+  bool initial_handshake_done;
+  bool reneg_started;
+};
+
+template <typename T> using EVCb = void (*)(struct ev_loop *, T *, int);
+
+using IOCb = EVCb<ev_io>;
+using TimerCb = EVCb<ev_timer>;
+
+struct Connection {
+  Connection(struct ev_loop *loop, int fd, SSL *ssl, ev_tstamp write_timeout,
+             ev_tstamp read_timeout, size_t write_rate, size_t write_burst,
+             size_t read_rate, size_t read_burst, IOCb writecb, IOCb readcb,
+             TimerCb timeoutcb, void *data);
+  ~Connection();
+
+  void disconnect();
+
+  int tls_handshake();
+
+  // All write_* and writev_clear functions return number of bytes
+  // written.  If nothing cannot be written (e.g., there is no
+  // allowance in RateLimit or underlying connection blocks), return
+  // 0.  SHRPX_ERR_NETWORK is returned in case of error.
+  //
+  // All read_* functions return number of bytes read.  If nothing
+  // cannot be read (e.g., there is no allowance in Ratelimit or
+  // underlying connection blocks), return 0.  SHRPX_ERR_EOF is
+  // returned in case of EOF and no data was read.  Otherwise
+  // SHRPX_ERR_NETWORK is return in case of error.
+  ssize_t write_tls(const void *data, size_t len);
+  ssize_t read_tls(void *data, size_t len);
+
+  size_t get_tls_write_limit();
+  // Updates the number of bytes written in warm up period.
+  void update_tls_warmup_writelen(size_t n);
+
+  ssize_t write_clear(const void *data, size_t len);
+  ssize_t writev_clear(struct iovec *iov, int iovcnt);
+  ssize_t read_clear(void *data, size_t len);
+
+  TLSConnection tls;
+  ev_io wev;
+  ev_io rev;
+  ev_timer wt;
+  ev_timer rt;
+  RateLimit wlimit;
+  RateLimit rlimit;
+  IOCb writecb;
+  IOCb readcb;
+  TimerCb timeoutcb;
+  struct ev_loop *loop;
+  void *data;
+  int fd;
+};
+
+} // namespace shrpx
+
+#endif // SHRPX_CONNECTION_H

src/shrpx_connection_handler.cc

@@ -32,13 +32,13 @@
 #include "shrpx_client_handler.h"
 #include "shrpx_ssl.h"
 #include "shrpx_worker.h"
-#include "shrpx_worker_config.h"
 #include "shrpx_config.h"
 #include "shrpx_http2_session.h"
 #include "shrpx_connect_blocker.h"
 #include "shrpx_downstream_connection.h"
 #include "shrpx_accept_handler.h"
 #include "util.h"
+#include "template.h"
 
 using namespace nghttp2;
 
@@ -50,7 +50,7 @@ void acceptor_disable_cb(struct ev_loop *loop, ev_timer *w, int revent) {
 
   // If we are in graceful shutdown period, we must not enable
   // acceptors again.
-  if (worker_config->graceful_shutdown) {
+  if (h->get_graceful_shutdown()) {
     return;
   }
 
@@ -59,25 +59,16 @@ void acceptor_disable_cb(struct ev_loop *loop, ev_timer *w, int revent) {
 } // namespace
 
 ConnectionHandler::ConnectionHandler(struct ev_loop *loop)
-    : loop_(loop), sv_ssl_ctx_(nullptr), cl_ssl_ctx_(nullptr),
-      // rate_limit_group_(bufferevent_rate_limit_group_new(
-      //     evbase, get_config()->worker_rate_limit_cfg)),
-      worker_stat_(util::make_unique<WorkerStat>()),
-      worker_round_robin_cnt_(0) {
+    : single_worker_(nullptr), loop_(loop), worker_round_robin_cnt_(0),
+      graceful_shutdown_(false) {
   ev_timer_init(&disable_acceptor_timer_, acceptor_disable_cb, 0., 0.);
   disable_acceptor_timer_.data = this;
 }
 
 ConnectionHandler::~ConnectionHandler() {
-  //  bufferevent_rate_limit_group_free(rate_limit_group_);
   ev_timer_stop(loop_, &disable_acceptor_timer_);
 }
 
-void ConnectionHandler::create_ssl_context() {
-  sv_ssl_ctx_ = ssl::setup_server_ssl_context();
-  cl_ssl_ctx_ = ssl::setup_client_ssl_context();
-}
-
 void ConnectionHandler::worker_reopen_log_files() {
   WorkerEvent wev;
 
@@ -102,14 +93,41 @@ void ConnectionHandler::worker_renew_ticket_keys(
   }
 }
 
+void ConnectionHandler::create_single_worker() {
+  auto cert_tree = ssl::create_cert_lookup_tree();
+  auto sv_ssl_ctx = ssl::setup_server_ssl_context(cert_tree);
+  auto cl_ssl_ctx = ssl::setup_client_ssl_context();
+
+  single_worker_ = make_unique<Worker>(loop_, sv_ssl_ctx, cl_ssl_ctx, cert_tree,
+                                       ticket_keys_);
+}
+
 void ConnectionHandler::create_worker_thread(size_t num) {
 #ifndef NOTHREADS
   assert(workers_.size() == 0);
 
+  SSL_CTX *sv_ssl_ctx = nullptr, *cl_ssl_ctx = nullptr;
+  ssl::CertLookupTree *cert_tree = nullptr;
+
+  if (!get_config()->tls_ctx_per_worker) {
+    cert_tree = ssl::create_cert_lookup_tree();
+    sv_ssl_ctx = ssl::setup_server_ssl_context(cert_tree);
+    cl_ssl_ctx = ssl::setup_client_ssl_context();
+  }
+
   for (size_t i = 0; i < num; ++i) {
-    workers_.push_back(util::make_unique<Worker>(sv_ssl_ctx_, cl_ssl_ctx_,
-                                                 worker_config->cert_tree,
-                                                 worker_config->ticket_keys));
+    auto loop = ev_loop_new(0);
+
+    if (get_config()->tls_ctx_per_worker) {
+      cert_tree = ssl::create_cert_lookup_tree();
+      sv_ssl_ctx = ssl::setup_server_ssl_context(cert_tree);
+      cl_ssl_ctx = ssl::setup_client_ssl_context();
+    }
+
+    auto worker = make_unique<Worker>(loop, sv_ssl_ctx, cl_ssl_ctx, cert_tree,
+                                      ticket_keys_);
+    worker->run_async();
+    workers_.push_back(std::move(worker));
 
     if (LOG_ENABLED(INFO)) {
       LLOG(INFO, this) << "Created thread #" << workers_.size() - 1;
@@ -163,7 +181,7 @@ int ConnectionHandler::handle_connection(int fd, sockaddr *addr, int addrlen) {
 
   if (get_config()->num_worker == 1) {
 
-    if (worker_stat_->num_connections >=
+    if (single_worker_->get_worker_stat()->num_connections >=
         get_config()->worker_frontend_connections) {
 
       if (LOG_ENABLED(INFO)) {
@@ -175,8 +193,8 @@ int ConnectionHandler::handle_connection(int fd, sockaddr *addr, int addrlen) {
       return -1;
     }
 
-    auto client = ssl::accept_connection(loop_, sv_ssl_ctx_, fd, addr, addrlen,
-                                         worker_stat_.get(), &dconn_pool_);
+    auto client =
+        ssl::accept_connection(single_worker_.get(), fd, addr, addrlen);
     if (!client) {
       LLOG(ERROR, this) << "ClientHandler creation failed";
 
@@ -184,13 +202,13 @@ int ConnectionHandler::handle_connection(int fd, sockaddr *addr, int addrlen) {
       return -1;
     }
 
-    client->set_http2_session(http2session_.get());
-    client->set_http1_connect_blocker(http1_connect_blocker_.get());
-
     return 0;
   }
 
   size_t idx = worker_round_robin_cnt_ % workers_.size();
+  if (LOG_ENABLED(INFO)) {
+    LOG(INFO) << "Dispatch connection to worker #" << idx;
+  }
   ++worker_round_robin_cnt_;
   WorkerEvent wev;
   memset(&wev, 0, sizeof(wev));
@@ -208,24 +226,16 @@ struct ev_loop *ConnectionHandler::get_loop() const {
   return loop_;
 }
 
-void ConnectionHandler::create_http2_session() {
-  http2session_ = util::make_unique<Http2Session>(loop_, cl_ssl_ctx_);
+Worker *ConnectionHandler::get_single_worker() const {
+  return single_worker_.get();
 }
 
-void ConnectionHandler::create_http1_connect_blocker() {
-  http1_connect_blocker_ = util::make_unique<ConnectBlocker>(loop_);
+void ConnectionHandler::set_acceptor(std::unique_ptr<AcceptHandler> h) {
+  acceptor_ = std::move(h);
 }
 
-const WorkerStat *ConnectionHandler::get_worker_stat() const {
-  return worker_stat_.get();
-}
-
-void ConnectionHandler::set_acceptor4(std::unique_ptr<AcceptHandler> h) {
-  acceptor4_ = std::move(h);
-}
-
-AcceptHandler *ConnectionHandler::get_acceptor4() const {
-  return acceptor4_.get();
+AcceptHandler *ConnectionHandler::get_acceptor() const {
+  return acceptor_.get();
 }
 
 void ConnectionHandler::set_acceptor6(std::unique_ptr<AcceptHandler> h) {
@@ -237,8 +247,8 @@ AcceptHandler *ConnectionHandler::get_acceptor6() const {
 }
 
 void ConnectionHandler::enable_acceptor() {
-  if (acceptor4_) {
-    acceptor4_->enable();
+  if (acceptor_) {
+    acceptor_->enable();
   }
 
   if (acceptor6_) {
@@ -247,8 +257,8 @@ void ConnectionHandler::enable_acceptor() {
 }
 
 void ConnectionHandler::disable_acceptor() {
-  if (acceptor4_) {
-    acceptor4_->disable();
+  if (acceptor_) {
+    acceptor_->disable();
   }
 
   if (acceptor6_) {
@@ -268,12 +278,35 @@ void ConnectionHandler::disable_acceptor_temporary(ev_tstamp t) {
 }
 
 void ConnectionHandler::accept_pending_connection() {
-  if (acceptor4_) {
-    acceptor4_->accept_connection();
+  if (acceptor_) {
+    acceptor_->accept_connection();
   }
   if (acceptor6_) {
     acceptor6_->accept_connection();
   }
 }
 
+void
+ConnectionHandler::set_ticket_keys(std::shared_ptr<TicketKeys> ticket_keys) {
+  ticket_keys_ = std::move(ticket_keys);
+  if (single_worker_) {
+    single_worker_->set_ticket_keys(ticket_keys_);
+  }
+}
+
+const std::shared_ptr<TicketKeys> &ConnectionHandler::get_ticket_keys() const {
+  return ticket_keys_;
+}
+
+void ConnectionHandler::set_graceful_shutdown(bool f) {
+  graceful_shutdown_ = f;
+  if (single_worker_) {
+    single_worker_->set_graceful_shutdown(f);
+  }
+}
+
+bool ConnectionHandler::get_graceful_shutdown() const {
+  return graceful_shutdown_;
+}
+
 } // namespace shrpx

src/shrpx_connection_handler.h

@@ -54,16 +54,19 @@ public:
   ConnectionHandler(struct ev_loop *loop);
   ~ConnectionHandler();
   int handle_connection(int fd, sockaddr *addr, int addrlen);
-  void create_ssl_context();
+  // Creates Worker object for single threaded configuration.
+  void create_single_worker();
+  // Creates |num| Worker objects for multi threaded configuration.
+  // The |num| must be strictly more than 1.
   void create_worker_thread(size_t num);
   void worker_reopen_log_files();
   void worker_renew_ticket_keys(const std::shared_ptr<TicketKeys> &ticket_keys);
+  void set_ticket_keys(std::shared_ptr<TicketKeys> ticket_keys);
+  const std::shared_ptr<TicketKeys> &get_ticket_keys() const;
   struct ev_loop *get_loop() const;
-  void create_http2_session();
-  void create_http1_connect_blocker();
-  const WorkerStat *get_worker_stat() const;
-  void set_acceptor4(std::unique_ptr<AcceptHandler> h);
-  AcceptHandler *get_acceptor4() const;
+  Worker *get_single_worker() const;
+  void set_acceptor(std::unique_ptr<AcceptHandler> h);
+  AcceptHandler *get_acceptor() const;
   void set_acceptor6(std::unique_ptr<AcceptHandler> h);
   AcceptHandler *get_acceptor6() const;
   void enable_acceptor();
@@ -71,26 +74,28 @@ public:
   void disable_acceptor_temporary(ev_tstamp t);
   void accept_pending_connection();
   void graceful_shutdown_worker();
+  void set_graceful_shutdown(bool f);
+  bool get_graceful_shutdown() const;
   void join_worker();
 
 private:
-  DownstreamConnectionPool dconn_pool_;
+  // Worker instances when multi threaded mode (-nN, N >= 2) is used.
   std::vector<std::unique_ptr<Worker>> workers_;
+  // Worker instance used when single threaded mode (-n1) is used.
+  // Otherwise, nullptr and workers_ has instances of Worker instead.
+  std::unique_ptr<Worker> single_worker_;
+  // Current TLS session ticket keys.  Note that TLS connection does
+  // not refer to this field directly.  They use TicketKeys object in
+  // Worker object.
+  std::shared_ptr<TicketKeys> ticket_keys_;
   struct ev_loop *loop_;
-  // The frontend server SSL_CTX
-  SSL_CTX *sv_ssl_ctx_;
-  // The backend server SSL_CTX
-  SSL_CTX *cl_ssl_ctx_;
-  // Shared backend HTTP2 session. NULL if multi-threaded. In
-  // multi-threaded case, see shrpx_worker.cc.
-  std::unique_ptr<Http2Session> http2session_;
-  std::unique_ptr<ConnectBlocker> http1_connect_blocker_;
-  // bufferevent_rate_limit_group *rate_limit_group_;
-  std::unique_ptr<AcceptHandler> acceptor4_;
+  // acceptor for IPv4 address or UNIX domain socket.
+  std::unique_ptr<AcceptHandler> acceptor_;
+  // acceptor for IPv6 address
   std::unique_ptr<AcceptHandler> acceptor6_;
   ev_timer disable_acceptor_timer_;
-  std::unique_ptr<WorkerStat> worker_stat_;
   unsigned int worker_round_robin_cnt_;
+  bool graceful_shutdown_;
 };
 
 } // namespace shrpx

src/shrpx_downstream.cc

@@ -112,8 +112,8 @@ Downstream::Downstream(Upstream *upstream, int32_t stream_id, int32_t priority)
       request_bodylen_(0), response_bodylen_(0), response_sent_bodylen_(0),
       request_content_length_(-1), response_content_length_(-1),
       upstream_(upstream), request_headers_sum_(0), response_headers_sum_(0),
-      request_datalen_(0), response_datalen_(0), stream_id_(stream_id),
-      priority_(priority), downstream_stream_id_(-1),
+      request_datalen_(0), response_datalen_(0), num_retry_(0),
+      stream_id_(stream_id), priority_(priority), downstream_stream_id_(-1),
       response_rst_stream_error_code_(NGHTTP2_NO_ERROR),
       request_state_(INITIAL), request_major_(1), request_minor_(1),
       response_state_(INITIAL), response_http_status_(0), response_major_(1),
@@ -122,7 +122,7 @@ Downstream::Downstream(Upstream *upstream, int32_t stream_id, int32_t priority)
       request_connection_close_(false), request_header_key_prev_(false),
       request_http2_expect_body_(false), chunked_response_(false),
       response_connection_close_(false), response_header_key_prev_(false),
-      expect_final_response_(false) {
+      expect_final_response_(false), request_pending_(false) {
 
   ev_timer_init(&upstream_rtimer_, &upstream_rtimeoutcb, 0.,
                 get_config()->stream_read_timeout);
@@ -238,7 +238,7 @@ void Downstream::assemble_request_cookie() {
   cookie = "";
   for (auto &kv : request_headers_) {
     if (kv.name.size() != 6 || kv.name[5] != 'e' ||
-        !util::streq("cooki", kv.name.c_str(), 5)) {
+        !util::streq_l("cooki", kv.name.c_str(), 5)) {
       continue;
     }
 
@@ -259,7 +259,7 @@ Headers Downstream::crumble_request_cookie() {
   Headers cookie_hdrs;
   for (auto &kv : request_headers_) {
     if (kv.name.size() != 6 || kv.name[5] != 'e' ||
-        !util::streq("cooki", kv.name.c_str(), 5)) {
+        !util::streq_l("cooki", kv.name.c_str(), 5)) {
       continue;
     }
     size_t last = kv.value.size();
@@ -288,7 +288,8 @@ const std::string &Downstream::get_assembled_request_cookie() const {
 }
 
 namespace {
-int index_headers(int *hdidx, Headers &headers, int64_t &content_length) {
+int index_headers(http2::HeaderIndex &hdidx, Headers &headers,
+                  int64_t &content_length) {
   for (size_t i = 0; i < headers.size(); ++i) {
     auto &kv = headers[i];
     util::inp_strlower(kv.name);
@@ -299,6 +300,7 @@ int index_headers(int *hdidx, Headers &headers, int64_t &content_length) {
       continue;
     }
 
+    kv.token = token;
     http2::index_header(hdidx, token, i);
 
     if (token == http2::HD_CONTENT_LENGTH) {
@@ -321,7 +323,7 @@ int Downstream::index_request_headers() {
                        request_content_length_);
 }
 
-const Headers::value_type *Downstream::get_request_header(int token) const {
+const Headers::value_type *Downstream::get_request_header(int16_t token) const {
   return http2::get_header(request_hdidx_, token, request_headers_);
 }
 
@@ -345,10 +347,11 @@ void Downstream::set_last_request_header_value(std::string value) {
 
 void Downstream::add_request_header(const uint8_t *name, size_t namelen,
                                     const uint8_t *value, size_t valuelen,
-                                    bool no_index, int token) {
+                                    bool no_index, int16_t token) {
   http2::index_header(request_hdidx_, token, request_headers_.size());
   request_headers_sum_ += namelen + valuelen;
-  http2::add_header(request_headers_, name, namelen, value, valuelen, no_index);
+  http2::add_header(request_headers_, name, namelen, value, valuelen, no_index,
+                    token);
 }
 
 bool Downstream::get_request_header_key_prev() const {
@@ -524,13 +527,13 @@ int Downstream::index_response_headers() {
                        response_content_length_);
 }
 
-const Headers::value_type *Downstream::get_response_header(int token) const {
+const Headers::value_type *
+Downstream::get_response_header(int16_t token) const {
   return http2::get_header(response_hdidx_, token, response_headers_);
 }
 
-void
-Downstream::rewrite_location_response_header(const std::string &upstream_scheme,
-                                             uint16_t upstream_port) {
+void Downstream::rewrite_location_response_header(
+    const std::string &upstream_scheme) {
   auto hd =
       http2::get_header(response_hdidx_, http2::HD_LOCATION, response_headers_);
   if (!hd) {
@@ -544,18 +547,43 @@ Downstream::rewrite_location_response_header(const std::string &upstream_scheme,
     return;
   }
   std::string new_uri;
-  if (!request_http2_authority_.empty()) {
-    new_uri =
-        http2::rewrite_location_uri((*hd).value, u, request_http2_authority_,
-                                    upstream_scheme, upstream_port);
-  }
-  if (new_uri.empty()) {
-    auto host = get_request_header(http2::HD_HOST);
-    if (!host) {
+  if (get_config()->no_host_rewrite) {
+    if (!request_http2_authority_.empty()) {
+      new_uri = http2::rewrite_location_uri(
+          (*hd).value, u, request_http2_authority_, request_http2_authority_,
+          upstream_scheme);
+    }
+    if (new_uri.empty()) {
+      auto host = get_request_header(http2::HD_HOST);
+      if (host) {
+        new_uri = http2::rewrite_location_uri((*hd).value, u, (*host).value,
+                                              (*host).value, upstream_scheme);
+      } else if (!request_downstream_host_.empty()) {
+        new_uri = http2::rewrite_location_uri(
+            (*hd).value, u, request_downstream_host_, "", upstream_scheme);
+      } else {
+        return;
+      }
+    }
+  } else {
+    if (request_downstream_host_.empty()) {
       return;
     }
-    new_uri = http2::rewrite_location_uri((*hd).value, u, (*host).value,
-                                          upstream_scheme, upstream_port);
+    if (!request_http2_authority_.empty()) {
+      new_uri = http2::rewrite_location_uri(
+          (*hd).value, u, request_downstream_host_, request_http2_authority_,
+          upstream_scheme);
+    } else {
+      auto host = get_request_header(http2::HD_HOST);
+      if (host) {
+        new_uri = http2::rewrite_location_uri((*hd).value, u,
+                                              request_downstream_host_,
+                                              (*host).value, upstream_scheme);
+      } else {
+        new_uri = http2::rewrite_location_uri(
+            (*hd).value, u, request_downstream_host_, "", upstream_scheme);
+      }
+    }
   }
   if (!new_uri.empty()) {
     auto idx = response_hdidx_[http2::HD_LOCATION];
@@ -576,13 +604,21 @@ void Downstream::set_last_response_header_value(std::string value) {
   item.value = std::move(value);
 }
 
+void Downstream::add_response_header(std::string name, std::string value,
+                                     int16_t token) {
+  http2::index_header(response_hdidx_, token, response_headers_.size());
+  response_headers_sum_ += name.size() + value.size();
+  response_headers_.emplace_back(std::move(name), std::move(value), false,
+                                 token);
+}
+
 void Downstream::add_response_header(const uint8_t *name, size_t namelen,
                                      const uint8_t *value, size_t valuelen,
-                                     bool no_index, int token) {
+                                     bool no_index, int16_t token) {
   http2::index_header(response_hdidx_, token, response_headers_.size());
   response_headers_sum_ += namelen + valuelen;
-  http2::add_header(response_headers_, name, namelen, value, valuelen,
-                    no_index);
+  http2::add_header(response_headers_, name, namelen, value, valuelen, no_index,
+                    token);
 }
 
 bool Downstream::get_response_header_key_prev() const {
@@ -777,8 +813,8 @@ void Downstream::inspect_http1_request() {
 
       auto &val = request_headers_[idx].value;
       // TODO Perform more strict checking for upgrade headers
-      if (util::streq(NGHTTP2_CLEARTEXT_PROTO_VERSION_ID, val.c_str(),
-                      val.size())) {
+      if (util::streq_l(NGHTTP2_CLEARTEXT_PROTO_VERSION_ID, val.c_str(),
+                        val.size())) {
         http2_upgrade_seen_ = true;
       }
     }
@@ -892,14 +928,14 @@ bool pseudo_header_allowed(const Headers &headers) {
 }
 } // namespace
 
-bool Downstream::request_pseudo_header_allowed(int token) const {
+bool Downstream::request_pseudo_header_allowed(int16_t token) const {
   if (!pseudo_header_allowed(request_headers_)) {
     return false;
   }
   return http2::check_http2_request_pseudo_header(request_hdidx_, token);
 }
 
-bool Downstream::response_pseudo_header_allowed(int token) const {
+bool Downstream::response_pseudo_header_allowed(int16_t token) const {
   if (!pseudo_header_allowed(response_headers_)) {
     return false;
   }
@@ -1020,4 +1056,22 @@ void Downstream::disable_downstream_wtimer() {
 
 bool Downstream::accesslog_ready() const { return response_http_status_ > 0; }
 
+void Downstream::add_retry() { ++num_retry_; }
+
+bool Downstream::no_more_retry() const { return num_retry_ > 5; }
+
+void Downstream::set_request_downstream_host(std::string host) {
+  request_downstream_host_ = std::move(host);
+}
+
+void Downstream::set_request_pending(bool f) { request_pending_ = f; }
+
+bool Downstream::get_request_pending() const { return request_pending_; }
+
+bool Downstream::request_submission_ready() const {
+  return (request_state_ == Downstream::HEADER_COMPLETE ||
+          request_state_ == Downstream::MSG_COMPLETE) &&
+         request_pending_ && response_state_ == Downstream::INITIAL;
+}
+
 } // namespace shrpx

src/shrpx_downstream.h

@@ -108,7 +108,7 @@ public:
   // multiple header have |name| as name, return last occurrence from
   // the beginning.  If no such header is found, returns nullptr.
   // This function must be called after headers are indexed
-  const Headers::value_type *get_request_header(int token) const;
+  const Headers::value_type *get_request_header(int16_t token) const;
   // Returns pointer to the request header with the name |name|.  If
   // no such header is found, returns nullptr.
   const Headers::value_type *get_request_header(const std::string &name) const;
@@ -117,7 +117,7 @@ public:
 
   void add_request_header(const uint8_t *name, size_t namelen,
                           const uint8_t *value, size_t valuelen, bool no_index,
-                          int token);
+                          int16_t token);
 
   bool get_request_header_key_prev() const;
   void append_last_request_header_key(const char *data, size_t len);
@@ -165,7 +165,8 @@ public:
   bool validate_request_bodylen() const;
   int64_t get_request_content_length() const;
   void set_request_content_length(int64_t len);
-  bool request_pseudo_header_allowed(int token) const;
+  bool request_pseudo_header_allowed(int16_t token) const;
+  void set_request_downstream_host(std::string host);
   bool expect_response_body() const;
   enum {
     INITIAL,
@@ -182,6 +183,10 @@ public:
   void set_request_state(int state);
   int get_request_state() const;
   DefaultMemchunks *get_request_buf();
+  void set_request_pending(bool f);
+  bool get_request_pending() const;
+  // Returns true if request is ready to be submitted to downstream.
+  bool request_submission_ready() const;
   // downstream response API
   const Headers &get_response_headers() const;
   // Lower the response header field names and indexes response
@@ -192,16 +197,16 @@ public:
   // multiple header have |name| as name, return last occurrence from
   // the beginning.  If no such header is found, returns nullptr.
   // This function must be called after response headers are indexed.
-  const Headers::value_type *get_response_header(int token) const;
+  const Headers::value_type *get_response_header(int16_t token) const;
   // Rewrites the location response header field.
-  void rewrite_location_response_header(const std::string &upstream_scheme,
-                                        uint16_t upstream_port);
+  void rewrite_location_response_header(const std::string &upstream_scheme);
   void add_response_header(std::string name, std::string value);
   void set_last_response_header_value(std::string value);
 
+  void add_response_header(std::string name, std::string value, int16_t token);
   void add_response_header(const uint8_t *name, size_t namelen,
                            const uint8_t *value, size_t valuelen, bool no_index,
-                           int token);
+                           int16_t token);
 
   bool get_response_header_key_prev() const;
   void append_last_response_header_key(const char *data, size_t len);
@@ -248,7 +253,7 @@ public:
   void dec_response_datalen(size_t len);
   size_t get_response_datalen() const;
   void reset_response_datalen();
-  bool response_pseudo_header_allowed(int token) const;
+  bool response_pseudo_header_allowed(int16_t token) const;
 
   // Call this method when there is incoming data in downstream
   // connection.
@@ -289,6 +294,11 @@ public:
   // Returns true if accesslog can be written for this downstream.
   bool accesslog_ready() const;
 
+  // Increment retry count
+  void add_retry();
+  // true if retry attempt should not be done.
+  bool no_more_retry() const;
+
   enum {
     EVENT_ERROR = 0x1,
     EVENT_TIMEOUT = 0x2,
@@ -304,6 +314,10 @@ private:
   std::string request_path_;
   std::string request_http2_scheme_;
   std::string request_http2_authority_;
+  // host we requested to downstream.  This is used to rewrite
+  // location header field to decide the location should be rewritten
+  // or not.
+  std::string request_downstream_host_;
   std::string assembled_request_cookie_;
 
   DefaultMemchunks request_buf_;
@@ -338,6 +352,8 @@ private:
   size_t request_datalen_;
   size_t response_datalen_;
 
+  size_t num_retry_;
+
   int32_t stream_id_;
   int32_t priority_;
   // stream ID in backend connection
@@ -355,8 +371,8 @@ private:
   int response_major_;
   int response_minor_;
 
-  int request_hdidx_[http2::HD_MAXIDX];
-  int response_hdidx_[http2::HD_MAXIDX];
+  http2::HeaderIndex request_hdidx_;
+  http2::HeaderIndex response_hdidx_;
 
   // true if the request contains upgrade token (HTTP Upgrade or
   // CONNECT)
@@ -375,6 +391,10 @@ private:
   bool response_connection_close_;
   bool response_header_key_prev_;
   bool expect_final_response_;
+  // true if downstream request is pending because backend connection
+  // has not been established or should be checked before use;
+  // currently used only with HTTP/2 connection.
+  bool request_pending_;
 };
 
 } // namespace shrpx

src/shrpx_downstream_connection.h

@@ -51,8 +51,6 @@ public:
   virtual int resume_read(IOCtrlReason reason, size_t consumed) = 0;
   virtual void force_resume_read() = 0;
 
-  enum { ERR_EOF = -100, ERR_NET = -101 };
-
   virtual int on_read() = 0;
   virtual int on_write() = 0;
   virtual int on_timeout() { return 0; }

src/shrpx_downstream_test.cc

@@ -136,20 +136,22 @@ void test_downstream_assemble_request_cookie(void) {
 void test_downstream_rewrite_location_response_header(void) {
   {
     Downstream d(nullptr, 0, 0);
-    d.add_request_header("host", "localhost:3000");
+    d.set_request_downstream_host("localhost:3000");
+    d.add_request_header("host", "localhost");
     d.add_response_header("location", "http://localhost:3000/");
     d.index_request_headers();
     d.index_response_headers();
-    d.rewrite_location_response_header("https", 443);
+    d.rewrite_location_response_header("https");
     auto location = d.get_response_header(http2::HD_LOCATION);
     CU_ASSERT("https://localhost/" == (*location).value);
   }
   {
     Downstream d(nullptr, 0, 0);
+    d.set_request_downstream_host("localhost");
     d.set_request_http2_authority("localhost");
-    d.add_response_header("location", "http://localhost/");
+    d.add_response_header("location", "http://localhost:3000/");
     d.index_response_headers();
-    d.rewrite_location_response_header("https", 443);
+    d.rewrite_location_response_header("https");
     auto location = d.get_response_header(http2::HD_LOCATION);
     CU_ASSERT("https://localhost/" == (*location).value);
   }

src/shrpx_error.h

@@ -29,11 +29,13 @@
 
 namespace shrpx {
 
+// Deprecated, do not use.
 enum ErrorCode {
   SHRPX_ERR_SUCCESS = 0,
-  SHRPX_ERR_UNKNOWN = -1,
-  SHRPX_ERR_HTTP_PARSE = -2,
-  SHRPX_ERR_NETWORK = -3
+  SHRPX_ERR_ERROR = -1,
+  SHRPX_ERR_NETWORK = -100,
+  SHRPX_ERR_EOF = -101,
+  SHRPX_ERR_INPROGRESS = -102,
 };
 
 } // namespace shrpx

src/shrpx_http.cc

@@ -26,7 +26,6 @@
 
 #include "shrpx_config.h"
 #include "shrpx_log.h"
-#include "shrpx_worker_config.h"
 #include "http2.h"
 #include "util.h"
 
@@ -55,8 +54,10 @@ std::string create_error_html(unsigned int status_code) {
 std::string create_via_header_value(int major, int minor) {
   std::string hdrs;
   hdrs += static_cast<char>(major + '0');
-  hdrs += ".";
-  hdrs += static_cast<char>(minor + '0');
+  if (major < 2) {
+    hdrs += ".";
+    hdrs += static_cast<char>(minor + '0');
+  }
   hdrs += " nghttpx";
   return hdrs;
 }

src/shrpx_http2_downstream_connection.cc

@@ -35,7 +35,6 @@
 #include "shrpx_error.h"
 #include "shrpx_http.h"
 #include "shrpx_http2_session.h"
-#include "shrpx_worker_config.h"
 #include "http2.h"
 #include "util.h"
 
@@ -222,16 +221,53 @@ ssize_t http2_data_read_callback(nghttp2_session *session, int32_t stream_id,
 
 int Http2DownstreamConnection::push_request_headers() {
   int rv;
+  if (!downstream_) {
+    return 0;
+  }
   if (!http2session_->can_push_request()) {
     // The HTTP2 session to the backend has not been established or
     // connection is now being checked.  This function will be called
     // again just after it is established.
+    downstream_->set_request_pending(true);
     http2session_->start_checking_connection();
     return 0;
   }
-  if (!downstream_) {
-    return 0;
+
+  downstream_->set_request_pending(false);
+
+  const char *authority = nullptr, *host = nullptr;
+  if (!get_config()->no_host_rewrite && !get_config()->http2_proxy &&
+      !get_config()->client_proxy) {
+    // HTTP/2 backend does not support multiple address, so we always
+    // use index = 0.
+    if (!downstream_->get_request_http2_authority().empty()) {
+      authority = get_config()->downstream_addrs[0].hostport.get();
+    }
+    if (downstream_->get_request_header(http2::HD_HOST)) {
+      host = get_config()->downstream_addrs[0].hostport.get();
+    }
+  } else {
+    if (!downstream_->get_request_http2_authority().empty()) {
+      authority = downstream_->get_request_http2_authority().c_str();
+    }
+    auto h = downstream_->get_request_header(http2::HD_HOST);
+    if (h) {
+      host = h->value.c_str();
+    }
   }
+
+  if (!authority && !host) {
+    // upstream is HTTP/1.0.  We use backend server's host
+    // nonetheless.
+    host = get_config()->downstream_addrs[0].hostport.get();
+  }
+
+  if (authority) {
+    downstream_->set_request_downstream_host(authority);
+  } else {
+    downstream_->set_request_downstream_host(host);
+  }
+
   size_t nheader = downstream_->get_request_headers().size();
 
   Headers cookies;
@@ -239,28 +275,28 @@ int Http2DownstreamConnection::push_request_headers() {
     cookies = downstream_->crumble_request_cookie();
   }
 
-  // 7 means:
+  // 8 means:
   // 1. :method
   // 2. :scheme
   // 3. :path
-  // 4. :authority (optional)
-  // 5. via (optional)
-  // 6. x-forwarded-for (optional)
-  // 7. x-forwarded-proto (optional)
+  // 4. :authority (at least either :authority or host exists)
+  // 5. host
+  // 6. via (optional)
+  // 7. x-forwarded-for (optional)
+  // 8. x-forwarded-proto (optional)
   auto nva = std::vector<nghttp2_nv>();
-  nva.reserve(nheader + 7 + cookies.size());
+  nva.reserve(nheader + 8 + cookies.size());
 
   std::string via_value;
   std::string xff_value;
-  std::string scheme, authority, path, query;
+  std::string scheme, uri_authority, path, query;
 
   // To reconstruct HTTP/1 status line and headers, proxy should
   // preserve host header field. See draft-09 section 8.1.3.1.
   if (downstream_->get_request_method() == "CONNECT") {
     // The upstream may be HTTP/2 or HTTP/1
-    if (!downstream_->get_request_http2_authority().empty()) {
-      nva.push_back(http2::make_nv_ls(
-          ":authority", downstream_->get_request_http2_authority()));
+    if (authority) {
+      nva.push_back(http2::make_nv_lc(":authority", authority));
     } else {
       nva.push_back(
           http2::make_nv_ls(":authority", downstream_->get_request_path()));
@@ -270,14 +306,8 @@ int Http2DownstreamConnection::push_request_headers() {
     nva.push_back(
         http2::make_nv_ls(":scheme", downstream_->get_request_http2_scheme()));
     nva.push_back(http2::make_nv_ls(":path", downstream_->get_request_path()));
-    if (!downstream_->get_request_http2_authority().empty()) {
-      nva.push_back(http2::make_nv_ls(
-          ":authority", downstream_->get_request_http2_authority()));
-    } else if (!downstream_->get_request_header(http2::HD_HOST)) {
-      if (LOG_ENABLED(INFO)) {
-        DCLOG(INFO, this) << "host header field missing";
-      }
-      return -1;
+    if (authority) {
+      nva.push_back(http2::make_nv_lc(":authority", authority));
     }
   } else {
     // The upstream is HTTP/1
@@ -288,11 +318,11 @@ int Http2DownstreamConnection::push_request_headers() {
                                &u);
     if (rv == 0) {
       http2::copy_url_component(scheme, &u, UF_SCHEMA, url);
-      http2::copy_url_component(authority, &u, UF_HOST, url);
+      http2::copy_url_component(uri_authority, &u, UF_HOST, url);
       http2::copy_url_component(path, &u, UF_PATH, url);
       http2::copy_url_component(query, &u, UF_QUERY, url);
       if (path.empty()) {
-        if (!authority.empty() &&
+        if (!uri_authority.empty() &&
             downstream_->get_request_method() == "OPTIONS") {
           path = "*";
         } else {
@@ -319,35 +349,40 @@ int Http2DownstreamConnection::push_request_headers() {
     } else {
       nva.push_back(http2::make_nv_ls(":path", path));
     }
-    if (!authority.empty()) {
+
+    if (!get_config()->no_host_rewrite && !get_config()->http2_proxy &&
+        !get_config()->client_proxy) {
+      if (authority) {
+        nva.push_back(http2::make_nv_lc(":authority", authority));
+      }
+    } else if (!uri_authority.empty()) {
       // TODO properly check IPv6 numeric address
-      if (authority.find(":") != std::string::npos) {
-        authority = "[" + authority;
-        authority += "]";
+      if (uri_authority.find(":") != std::string::npos) {
+        uri_authority = "[" + uri_authority;
+        uri_authority += "]";
       }
       if (u.field_set & (1 << UF_PORT)) {
-        authority += ":";
-        authority += util::utos(u.port);
+        uri_authority += ":";
+        uri_authority += util::utos(u.port);
       }
-      nva.push_back(http2::make_nv_ls(":authority", authority));
-    } else if (!downstream_->get_request_header(http2::HD_HOST)) {
-      if (LOG_ENABLED(INFO)) {
-        DCLOG(INFO, this) << "host header field missing";
-      }
-      return -1;
+      nva.push_back(http2::make_nv_ls(":authority", uri_authority));
     }
   }
 
   nva.push_back(
       http2::make_nv_ls(":method", downstream_->get_request_method()));
 
+  if (host) {
+    nva.push_back(http2::make_nv_lc("host", host));
+  }
+
   http2::copy_headers_to_nva(nva, downstream_->get_request_headers());
 
   bool chunked_encoding = false;
   auto transfer_encoding =
       downstream_->get_request_header(http2::HD_TRANSFER_ENCODING);
   if (transfer_encoding &&
-      util::strieq((*transfer_encoding).value.c_str(), "chunked")) {
+      util::strieq_l("chunked", (*transfer_encoding).value)) {
     chunked_encoding = true;
   }