Get Started with Streaming Rendering in 5 Minutes

Streaming Rendering#

First, a brief introduction to streaming rendering. Streaming rendering leverages the chunked transfer encoding feature of HTTP/1.1 to allow the server to return HTML in chunks. The browser can then progressively render the page as it receives these chunks, which helps with the first meaningful paint (FMP) and enhances the user experience.

Chunked Transfer Encoding#

Chunked transfer encoding is a data transfer mechanism in the Hypertext Transfer Protocol (HTTP) that allows HTTP data sent from a web server to a client application (usually a web browser) to be divided into multiple parts. Chunked transfer encoding is only available in HTTP Protocol version 1.1 (HTTP/1.1).

Chunked Transfer Format#

If the Transfer-Encoding message header in an HTTP message (including request messages sent by the client or response messages returned by the server) has a value of chunked, then the message body consists of an unspecified number of chunks, ending with a chunk of size 0. Each non-empty chunk begins with the byte count of the data contained in that chunk (the byte count is expressed in hexadecimal), followed by a CRLF (carriage return and line feed), then the data itself, and finally ends with a CRLF. In some implementations, whitespace (0x20) is padded between the chunk size and the CRLF. The last chunk is a single line, consisting of the chunk size (0), some optional padding whitespace, and a CRLF. The last chunk contains no data but may send optional trailers, including message header fields. The message ends with a CRLF.

An Example#

Both the mobile and PC versions of our store underwent streaming rendering optimizations, resulting in a first-screen improvement of around 200ms and corresponding increases in business metrics.

Sequence Diagram Comparing Regular and Streaming Rendering for Stores:

Engineering Solution: Spark#

While the benefits of streaming rendering are clear, integrating it into each application still incurs some cost. I decided to extract a solution from the store’s optimization approach that would allow for rapid integration of streaming rendering.

Solution Research#

Streaming rendering itself is quite simple: the backend just needs to output HTML in chunks. The challenge lies in how to enable business developers to integrate it efficiently and painlessly. I initially considered a few solutions and reviewed them with the team during our weekly meeting:

Solution A#

Provide a middleware or plugin that implements the streaming rendering logic.

The caller integrates the middleware and uses a manual configuration approach to associate templates with data. The server (Node or Java) then streams the page based on this configuration.

For example, using egg.js:

xxx Controller {
  test() {
    //  spark 为流式渲染中间件挂载在 ctx 上的方法
    const pipe = this.ctx.spark();
    this.ctx.body = pipe;
    //  设置分块渲染的配置
    pipe.setPageConfig([{
      tpl: 'admin/index-pipe/pipe-1.nj',
      getData: (ctx) => Promise.resolve(mergerCommonContext({}, ctx)),
    }, {
      tpl: 'admin/index-pipe/pipe-2.nj',
      getData: ctx => bindGetPageData(ctx),
    }]);
    // 调用流式渲染方法
    pipe.render()
  }
}

Pros:

Low implementation cost
Minimal business intrusion: no need to modify business logic

Cons:

Requires template splitting

Solution B#

Achieve page chunking through template conventions. Then, when the server renders the template, data is inserted into the template chunks to enable chunked rendering.

Example: Template

<!-- spark:start block0 -->
<!doctype html>
<html>
  <head>
    <title>Test</title>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="renderer" content="webkit" />
    <meta
      name="viewport"
      content="width=device-width, initial-scale=1.0, user-scalable=no"
    />
    <link
      rel="shortcut icon"
      href="//is.alicdn.com/favicon.ico"
      type="image/x-icon"
    />
  </head>
  <body>
    <!-- spark:end block0 -->

    <!-- spark:start block1 -->
    <div style="width: 100%">
      {% for item in pageData %}
      <img
        src="http://design3d-zhangbei.oss-cn-zhangjiakou.aliyuncs.com/{{ item.thumbnailOssKey }}"
        style="
          width: 200px;
          height: 200px;
          display: inline-block;
          vertical-align: top;
        "
      />
      {% endfor %}
    </div>
    <!-- spark:end block1 -->

    <!-- spark:start block2 -->
    <div style="width: 100%">
      {% for item in pageData %}
      <img
        src="http://design3d-zhangbei.oss-cn-zhangjiakou.aliyuncs.com/{{ item.thumbnailOssKey }}"
        style="
          width: 200px;
          height: 200px;
          display: inline-block;
          vertical-align: top;
        "
      />
      {% endfor %}
    </div>
    <!-- spark:end block2 -->

    <!-- spark:start block3 -->

    <script>
      window._PAGE_DATA_ = {{ pageData | default({}) | dump | replace(r/<\/script/ig, '<\\\/script') | safe }};
    </script>
  </body>
</html>
<!-- spark:start block3 -->

Server-side call:

xxx Controller {
  test() {
    //  spark 为流式渲染中间件挂载在 ctx 上的方法
    const pipe = this.ctx.spark();
    this.ctx.body = pipe;
    //  设置分块渲染的配置
    pipe.renderBlock( 'block0', {xxx})
    pipe.renderBlock( 'block1', {xxx})
    pipe.renderBlock( 'block2', {xxx})
    pipe.renderBlock( 'block3', {xxx})
  }
}

BigPipe#

Streaming rendering still relies on the server’s return order. When multiple page blocks need data fetching and rendering, they must be called and returned serially. Taking streaming rendering a step further leads to BigPipe, which eliminates the dependency on the backend’s return order. Whichever block finishes data fetching and rendering first, its HTML is returned to the browser for rendering.

The principle is very simple: the server first returns the page layout. Then, as each page block is rendered, it is returned to the browser in the form of a script tag, for example:

<script>
  spark.render("#id", "<div>page1</div>");
</script>

By using a method agreed upon with the frontend, JavaScript is used to insert HTML into specific parts of the page, achieving progressive rendering. Pros:

Can return content for browser rendering faster, without being bound by HTML order. Cons:
Inline script tags within HTML inserted by JavaScript will not execute. For example, the onclick event below will not trigger:

<script>
  spark.render('#id', '<div>page1<script>console.log(123)</script></div>');
</script>

The content of the script tag needs to be extracted and evaluated separately to execute.
Requires intrusion into frontend code: since pages are returned out of order, the original mechanism that relied on script insertion order to guarantee execution sequence might fail. This necessitates using events or other methods to notify page rendering status to ensure JS execution order, increasing complexity.

Conclusion#

After discussing with colleagues, the main suggestions received were:

Although Solution B was preferred over A, there was still resistance to modifying the template itself.
For some businesses, a single template serves as a layout for multiple business scenarios. If different page splitting is required, Solution B presents problems.
The demand for BigPipe was not strong, as the performance improvement was limited while the cost was relatively high.

Based on these opinions, the original solution was improved:

Templates do not need to be concerned with whether they are used for streaming rendered pages.
Support for BigPipe was removed.

The final solution is as follows:

First, call the SparkRender method provided by the plugin, passing regList and dataList, and immediately return a Readable Stream to the browser.
The HTML is split according to regList, then combined with dataList to form a blockQueue awaiting rendering.
Following the order of the blockQueue, the RenderBlock method is called serially, and the rendering results are pushed to the Readable Stream.
When the blockQueue has finished rendering, the stream is closed, and the entire streaming rendering process ends.

Support Status#

Currently, it only supports Node applications based on egg.js.

Applicable Scenarios#

All scenarios requiring backend template rendering with a focus on performance.

How to Use#

For an egg.js-based Node application, only two steps are needed for integration:

Introduce and enable the egg-spark plugin in your application:

tnpm install egg-spark

在 `config/plugin.js` 中开启插件
exports.spark = {
  enable: true,
  package: 'egg-spark',
};

Make a small modification to the controller for the page you want to enable streaming rendering: replace the original render method with the spark plugin’s render: spark.render(tpl, regList, dataList):void

Parameters:

Parameter	Type	Meaning
`tplPath`	`string`	The template address to render, same meaning as the framework’s parameter, no modification needed.
`regList`	`reg[]`	A list of regular expressions used to split the page. The page is split into `regList.length + 1` parts. Each split is based on the first character matched by the current regex.
`dataList`	`object[]` / `function[]`	Data to be passed for rendering the corresponding template blocks based on the `regList` splits. Can be a data `object` directly, or a synchronous or asynchronous data function.

Example:

//原代码：
this.ctx.render("admin/index.nj", mergerCommonContext({ pageData }, this.ctx));
//修改后代码：
this.ctx.spark.render("admin/index.nj", {
  // regList 是一个用来切分页面的正则列表，根据 regList 的长度将页面切分为 regList.length+1 份
  regList: [/id="root"/],
  // dataList：根据 regList 切分的页面块，来传入对应的渲染模板需要的数据。可以传入数据 object ，也可以传入同步或者异步的数据方法
  dataList: [
    mergerCommonContext({}, this.ctx),
    async () => getPageData.call(this),
  ],
});

Thus your page already supports streaming rendering. Yes, it’s that simple.

Best Practices#

Generally, a page only needs to be split into two blocks:

First, return the <head> section containing the page’s css tags, so the browser can start loading and parsing css resources.
Simultaneously, the backend begins other time-consuming operations like data fetching.

Taking the recently launched custom page as an example, let’s see how to integrate streaming rendering with Spark. This is the template for the custom admin backend, admin/index.nj:

<!doctype html>
<html>
  <head>
    <script>
      window._customizePerfTimeTTfb = Date.now();
    </script>
    ...
    <link
      rel="shortcut icon"
      href="//is.alicdn.com/favicon.ico"
      type="image/x-icon"
    />
    {% block cssContents %}
    <link rel="stylesheet" href="{{cssFile}}" />
    {% endblock %}
    <script>
      window._customizePerfTimeCss = Date.now();
    </script>
  </head>
  <body data-spm="customize" class="{{ ctx.session.locale }}">
    {% block header %}
    <div class="page-header">{% using "mm-sc-new-header-v5" %}</div>
    {% endblock %}
    <div class="page-main">
      <div class="content">
        {% block content %}
        <div class="page-root" id="root"></div>
        <script>
          window._PAGE_DATA_ = {{ pageData | default({}) | dump | replace(r/<\/script/ig, '<\\\/script') | safe }};
          {% if pageConfig %}
          window._PAGE_CONFIG_ = {{ pageConfig | default({}) | dump | safe }};
          {% endif %}
        </script>
        {% endblock %}
      </div>
    </div>
  </body>
</html>

First, return the content of the <head> section, then return the <body>. The <head> only depends on static resource versions and takes almost no time, while the <body> depends on some relatively time-consuming data fetching logic. All that needs to be done is to modify the admin page’s controller:

module.exports = app => class AdminController extends app.Controller {
  async index() {
    // 首先对用户进行鉴权
    const previliges = await this.service.userInfo.checkPrevilige();
		// 调用插件提供的 rende 方法
    this.ctx.spark.render('admin/index.nj', {
      // 要先返回 head 部分，所以正则需要匹配 head 紧跟着的第一个元素，也就是 <body
      regList: [/<body data-spm="customize"/],
      dataList: [
        // 由于第一部分需要的数据是现成的，直接挂载在 ctx 上，所以直接传入就好了
        this.ctx,
        // 第二部分是复杂的取数逻辑，将取数的逻辑封装成一个异步方法传进来就好了
        async () => getPageData.call(this),
      ],
    });

    async function getPageData() {
      ...耗时逻辑
      const pageData = {
        ...
      };
      return mergerCommonContext({ pageData }, this.ctx);
    }
  }
};

This way, when the browser requests the admin page, it will be split and streamed according to the given regex.

Caveats#

Always perform authentication on the request before calling the render method, because once the page starts returning, it’s too late to perform authentication.

Summary#

Initially, I wasn’t very enthusiastic about doing this, but after it was built, it was successfully integrated into two businesses within half a day, and it led to improved page performance. The results were quite satisfying.