buddy

node MVC discord bot
Log | Files | Refs | README

README.md (19782B)


      1 node-fetch
      2 ==========
      3 
      4 [![npm version][npm-image]][npm-url]
      5 [![build status][travis-image]][travis-url]
      6 [![coverage status][codecov-image]][codecov-url]
      7 [![install size][install-size-image]][install-size-url]
      8 [![Discord][discord-image]][discord-url]
      9 
     10 A light-weight module that brings `window.fetch` to Node.js
     11 
     12 (We are looking for [v2 maintainers and collaborators](https://github.com/bitinn/node-fetch/issues/567))
     13 
     14 [![Backers][opencollective-image]][opencollective-url]
     15 
     16 <!-- TOC -->
     17 
     18 - [Motivation](#motivation)
     19 - [Features](#features)
     20 - [Difference from client-side fetch](#difference-from-client-side-fetch)
     21 - [Installation](#installation)
     22 - [Loading and configuring the module](#loading-and-configuring-the-module)
     23 - [Common Usage](#common-usage)
     24     - [Plain text or HTML](#plain-text-or-html)
     25     - [JSON](#json)
     26     - [Simple Post](#simple-post)
     27     - [Post with JSON](#post-with-json)
     28     - [Post with form parameters](#post-with-form-parameters)
     29     - [Handling exceptions](#handling-exceptions)
     30     - [Handling client and server errors](#handling-client-and-server-errors)
     31 - [Advanced Usage](#advanced-usage)
     32     - [Streams](#streams)
     33     - [Buffer](#buffer)
     34     - [Accessing Headers and other Meta data](#accessing-headers-and-other-meta-data)
     35     - [Extract Set-Cookie Header](#extract-set-cookie-header)
     36     - [Post data using a file stream](#post-data-using-a-file-stream)
     37     - [Post with form-data (detect multipart)](#post-with-form-data-detect-multipart)
     38     - [Request cancellation with AbortSignal](#request-cancellation-with-abortsignal)
     39 - [API](#api)
     40     - [fetch(url[, options])](#fetchurl-options)
     41     - [Options](#options)
     42     - [Class: Request](#class-request)
     43     - [Class: Response](#class-response)
     44     - [Class: Headers](#class-headers)
     45     - [Interface: Body](#interface-body)
     46     - [Class: FetchError](#class-fetcherror)
     47 - [License](#license)
     48 - [Acknowledgement](#acknowledgement)
     49 
     50 <!-- /TOC -->
     51 
     52 ## Motivation
     53 
     54 Instead of implementing `XMLHttpRequest` in Node.js to run browser-specific [Fetch polyfill](https://github.com/github/fetch), why not go from native `http` to `fetch` API directly? Hence, `node-fetch`, minimal code for a `window.fetch` compatible API on Node.js runtime.
     55 
     56 See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorphic-fetch) or Leonardo Quixada's [cross-fetch](https://github.com/lquixada/cross-fetch) for isomorphic usage (exports `node-fetch` for server-side, `whatwg-fetch` for client-side).
     57 
     58 ## Features
     59 
     60 - Stay consistent with `window.fetch` API.
     61 - Make conscious trade-off when following [WHATWG fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known differences.
     62 - Use native promise but allow substituting it with [insert your favorite promise library].
     63 - Use native Node streams for body on both request and response.
     64 - Decode content encoding (gzip/deflate) properly and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically.
     65 - Useful extensions such as timeout, redirect limit, response size limit, [explicit errors](ERROR-HANDLING.md) for troubleshooting.
     66 
     67 ## Difference from client-side fetch
     68 
     69 - See [Known Differences](LIMITS.md) for details.
     70 - If you happen to use a missing feature that `window.fetch` offers, feel free to open an issue.
     71 - Pull requests are welcomed too!
     72 
     73 ## Installation
     74 
     75 Current stable release (`2.x`)
     76 
     77 ```sh
     78 $ npm install node-fetch
     79 ```
     80 
     81 ## Loading and configuring the module
     82 We suggest you load the module via `require` until the stabilization of ES modules in node:
     83 ```js
     84 const fetch = require('node-fetch');
     85 ```
     86 
     87 If you are using a Promise library other than native, set it through `fetch.Promise`:
     88 ```js
     89 const Bluebird = require('bluebird');
     90 
     91 fetch.Promise = Bluebird;
     92 ```
     93 
     94 ## Common Usage
     95 
     96 NOTE: The documentation below is up-to-date with `2.x` releases; see the [`1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide](UPGRADE-GUIDE.md) for the differences.
     97 
     98 #### Plain text or HTML
     99 ```js
    100 fetch('https://github.com/')
    101     .then(res => res.text())
    102     .then(body => console.log(body));
    103 ```
    104 
    105 #### JSON
    106 
    107 ```js
    108 
    109 fetch('https://api.github.com/users/github')
    110     .then(res => res.json())
    111     .then(json => console.log(json));
    112 ```
    113 
    114 #### Simple Post
    115 ```js
    116 fetch('https://httpbin.org/post', { method: 'POST', body: 'a=1' })
    117     .then(res => res.json()) // expecting a json response
    118     .then(json => console.log(json));
    119 ```
    120 
    121 #### Post with JSON
    122 
    123 ```js
    124 const body = { a: 1 };
    125 
    126 fetch('https://httpbin.org/post', {
    127         method: 'post',
    128         body:    JSON.stringify(body),
    129         headers: { 'Content-Type': 'application/json' },
    130     })
    131     .then(res => res.json())
    132     .then(json => console.log(json));
    133 ```
    134 
    135 #### Post with form parameters
    136 `URLSearchParams` is available in Node.js as of v7.5.0. See [official documentation](https://nodejs.org/api/url.html#url_class_urlsearchparams) for more usage methods.
    137 
    138 NOTE: The `Content-Type` header is only set automatically to `x-www-form-urlencoded` when an instance of `URLSearchParams` is given as such:
    139 
    140 ```js
    141 const { URLSearchParams } = require('url');
    142 
    143 const params = new URLSearchParams();
    144 params.append('a', 1);
    145 
    146 fetch('https://httpbin.org/post', { method: 'POST', body: params })
    147     .then(res => res.json())
    148     .then(json => console.log(json));
    149 ```
    150 
    151 #### Handling exceptions
    152 NOTE: 3xx-5xx responses are *NOT* exceptions and should be handled in `then()`; see the next section for more information.
    153 
    154 Adding a catch to the fetch promise chain will catch *all* exceptions, such as errors originating from node core libraries, network errors and operational errors, which are instances of FetchError. See the [error handling document](ERROR-HANDLING.md)  for more details.
    155 
    156 ```js
    157 fetch('https://domain.invalid/')
    158     .catch(err => console.error(err));
    159 ```
    160 
    161 #### Handling client and server errors
    162 It is common to create a helper function to check that the response contains no client (4xx) or server (5xx) error responses:
    163 
    164 ```js
    165 function checkStatus(res) {
    166     if (res.ok) { // res.status >= 200 && res.status < 300
    167         return res;
    168     } else {
    169         throw MyCustomError(res.statusText);
    170     }
    171 }
    172 
    173 fetch('https://httpbin.org/status/400')
    174     .then(checkStatus)
    175     .then(res => console.log('will not get here...'))
    176 ```
    177 
    178 ## Advanced Usage
    179 
    180 #### Streams
    181 The "Node.js way" is to use streams when possible:
    182 
    183 ```js
    184 fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png')
    185     .then(res => {
    186         const dest = fs.createWriteStream('./octocat.png');
    187         res.body.pipe(dest);
    188     });
    189 ```
    190 
    191 #### Buffer
    192 If you prefer to cache binary data in full, use buffer(). (NOTE: `buffer()` is a `node-fetch`-only API)
    193 
    194 ```js
    195 const fileType = require('file-type');
    196 
    197 fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png')
    198     .then(res => res.buffer())
    199     .then(buffer => fileType(buffer))
    200     .then(type => { /* ... */ });
    201 ```
    202 
    203 #### Accessing Headers and other Meta data
    204 ```js
    205 fetch('https://github.com/')
    206     .then(res => {
    207         console.log(res.ok);
    208         console.log(res.status);
    209         console.log(res.statusText);
    210         console.log(res.headers.raw());
    211         console.log(res.headers.get('content-type'));
    212     });
    213 ```
    214 
    215 #### Extract Set-Cookie Header
    216 
    217 Unlike browsers, you can access raw `Set-Cookie` headers manually using `Headers.raw()`. This is a `node-fetch` only API.
    218 
    219 ```js
    220 fetch(url).then(res => {
    221     // returns an array of values, instead of a string of comma-separated values
    222     console.log(res.headers.raw()['set-cookie']);
    223 });
    224 ```
    225 
    226 #### Post data using a file stream
    227 
    228 ```js
    229 const { createReadStream } = require('fs');
    230 
    231 const stream = createReadStream('input.txt');
    232 
    233 fetch('https://httpbin.org/post', { method: 'POST', body: stream })
    234     .then(res => res.json())
    235     .then(json => console.log(json));
    236 ```
    237 
    238 #### Post with form-data (detect multipart)
    239 
    240 ```js
    241 const FormData = require('form-data');
    242 
    243 const form = new FormData();
    244 form.append('a', 1);
    245 
    246 fetch('https://httpbin.org/post', { method: 'POST', body: form })
    247     .then(res => res.json())
    248     .then(json => console.log(json));
    249 
    250 // OR, using custom headers
    251 // NOTE: getHeaders() is non-standard API
    252 
    253 const form = new FormData();
    254 form.append('a', 1);
    255 
    256 const options = {
    257     method: 'POST',
    258     body: form,
    259     headers: form.getHeaders()
    260 }
    261 
    262 fetch('https://httpbin.org/post', options)
    263     .then(res => res.json())
    264     .then(json => console.log(json));
    265 ```
    266 
    267 #### Request cancellation with AbortSignal
    268 
    269 > NOTE: You may cancel streamed requests only on Node >= v8.0.0
    270 
    271 You may cancel requests with `AbortController`. A suggested implementation is [`abort-controller`](https://www.npmjs.com/package/abort-controller).
    272 
    273 An example of timing out a request after 150ms could be achieved as the following:
    274 
    275 ```js
    276 import AbortController from 'abort-controller';
    277 
    278 const controller = new AbortController();
    279 const timeout = setTimeout(
    280   () => { controller.abort(); },
    281   150,
    282 );
    283 
    284 fetch(url, { signal: controller.signal })
    285   .then(res => res.json())
    286   .then(
    287     data => {
    288       useData(data)
    289     },
    290     err => {
    291       if (err.name === 'AbortError') {
    292         // request was aborted
    293       }
    294     },
    295   )
    296   .finally(() => {
    297     clearTimeout(timeout);
    298   });
    299 ```
    300 
    301 See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for more examples.
    302 
    303 
    304 ## API
    305 
    306 ### fetch(url[, options])
    307 
    308 - `url` A string representing the URL for fetching
    309 - `options` [Options](#fetch-options) for the HTTP(S) request
    310 - Returns: <code>Promise&lt;[Response](#class-response)&gt;</code>
    311 
    312 Perform an HTTP(S) fetch.
    313 
    314 `url` should be an absolute url, such as `https://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected `Promise`.
    315 
    316 <a id="fetch-options"></a>
    317 ### Options
    318 
    319 The default values are shown after each option key.
    320 
    321 ```js
    322 {
    323     // These properties are part of the Fetch Standard
    324     method: 'GET',
    325     headers: {},        // request headers. format is the identical to that accepted by the Headers constructor (see below)
    326     body: null,         // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream
    327     redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect
    328     signal: null,       // pass an instance of AbortSignal to optionally abort requests
    329 
    330     // The following properties are node-fetch extensions
    331     follow: 20,         // maximum redirect count. 0 to not follow redirect
    332     timeout: 0,         // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies). Signal is recommended instead.
    333     compress: true,     // support gzip/deflate content encoding. false to disable
    334     size: 0,            // maximum response body size in bytes. 0 to disable
    335     agent: null         // http(s).Agent instance or function that returns an instance (see below)
    336 }
    337 ```
    338 
    339 ##### Default Headers
    340 
    341 If no values are set, the following request headers will be sent automatically:
    342 
    343 Header              | Value
    344 ------------------- | --------------------------------------------------------
    345 `Accept-Encoding`   | `gzip,deflate` _(when `options.compress === true`)_
    346 `Accept`            | `*/*`
    347 `Connection`        | `close` _(when no `options.agent` is present)_
    348 `Content-Length`    | _(automatically calculated, if possible)_
    349 `Transfer-Encoding` | `chunked` _(when `req.body` is a stream)_
    350 `User-Agent`        | `node-fetch/1.0 (+https://github.com/bitinn/node-fetch)`
    351 
    352 Note: when `body` is a `Stream`, `Content-Length` is not set automatically.
    353 
    354 ##### Custom Agent
    355 
    356 The `agent` option allows you to specify networking related options which are out of the scope of Fetch, including and not limited to the following:
    357 
    358 - Support self-signed certificate
    359 - Use only IPv4 or IPv6
    360 - Custom DNS Lookup
    361 
    362 See [`http.Agent`](https://nodejs.org/api/http.html#http_new_agent_options) for more information.
    363 
    364 In addition, the `agent` option accepts a function that returns `http`(s)`.Agent` instance given current [URL](https://nodejs.org/api/url.html), this is useful during a redirection chain across HTTP and HTTPS protocol.
    365 
    366 ```js
    367 const httpAgent = new http.Agent({
    368     keepAlive: true
    369 });
    370 const httpsAgent = new https.Agent({
    371     keepAlive: true
    372 });
    373 
    374 const options = {
    375     agent: function (_parsedURL) {
    376         if (_parsedURL.protocol == 'http:') {
    377             return httpAgent;
    378         } else {
    379             return httpsAgent;
    380         }
    381     }
    382 }
    383 ```
    384 
    385 <a id="class-request"></a>
    386 ### Class: Request
    387 
    388 An HTTP(S) request containing information about URL, method, headers, and the body. This class implements the [Body](#iface-body) interface.
    389 
    390 Due to the nature of Node.js, the following properties are not implemented at this moment:
    391 
    392 - `type`
    393 - `destination`
    394 - `referrer`
    395 - `referrerPolicy`
    396 - `mode`
    397 - `credentials`
    398 - `cache`
    399 - `integrity`
    400 - `keepalive`
    401 
    402 The following node-fetch extension properties are provided:
    403 
    404 - `follow`
    405 - `compress`
    406 - `counter`
    407 - `agent`
    408 
    409 See [options](#fetch-options) for exact meaning of these extensions.
    410 
    411 #### new Request(input[, options])
    412 
    413 <small>*(spec-compliant)*</small>
    414 
    415 - `input` A string representing a URL, or another `Request` (which will be cloned)
    416 - `options` [Options][#fetch-options] for the HTTP(S) request
    417 
    418 Constructs a new `Request` object. The constructor is identical to that in the [browser](https://developer.mozilla.org/en-US/docs/Web/API/Request/Request).
    419 
    420 In most cases, directly `fetch(url, options)` is simpler than creating a `Request` object.
    421 
    422 <a id="class-response"></a>
    423 ### Class: Response
    424 
    425 An HTTP(S) response. This class implements the [Body](#iface-body) interface.
    426 
    427 The following properties are not implemented in node-fetch at this moment:
    428 
    429 - `Response.error()`
    430 - `Response.redirect()`
    431 - `type`
    432 - `trailer`
    433 
    434 #### new Response([body[, options]])
    435 
    436 <small>*(spec-compliant)*</small>
    437 
    438 - `body` A `String` or [`Readable` stream][node-readable]
    439 - `options` A [`ResponseInit`][response-init] options dictionary
    440 
    441 Constructs a new `Response` object. The constructor is identical to that in the [browser](https://developer.mozilla.org/en-US/docs/Web/API/Response/Response).
    442 
    443 Because Node.js does not implement service workers (for which this class was designed), one rarely has to construct a `Response` directly.
    444 
    445 #### response.ok
    446 
    447 <small>*(spec-compliant)*</small>
    448 
    449 Convenience property representing if the request ended normally. Will evaluate to true if the response status was greater than or equal to 200 but smaller than 300.
    450 
    451 #### response.redirected
    452 
    453 <small>*(spec-compliant)*</small>
    454 
    455 Convenience property representing if the request has been redirected at least once. Will evaluate to true if the internal redirect counter is greater than 0.
    456 
    457 <a id="class-headers"></a>
    458 ### Class: Headers
    459 
    460 This class allows manipulating and iterating over a set of HTTP headers. All methods specified in the [Fetch Standard][whatwg-fetch] are implemented.
    461 
    462 #### new Headers([init])
    463 
    464 <small>*(spec-compliant)*</small>
    465 
    466 - `init` Optional argument to pre-fill the `Headers` object
    467 
    468 Construct a new `Headers` object. `init` can be either `null`, a `Headers` object, an key-value map object or any iterable object.
    469 
    470 ```js
    471 // Example adapted from https://fetch.spec.whatwg.org/#example-headers-class
    472 
    473 const meta = {
    474   'Content-Type': 'text/xml',
    475   'Breaking-Bad': '<3'
    476 };
    477 const headers = new Headers(meta);
    478 
    479 // The above is equivalent to
    480 const meta = [
    481   [ 'Content-Type', 'text/xml' ],
    482   [ 'Breaking-Bad', '<3' ]
    483 ];
    484 const headers = new Headers(meta);
    485 
    486 // You can in fact use any iterable objects, like a Map or even another Headers
    487 const meta = new Map();
    488 meta.set('Content-Type', 'text/xml');
    489 meta.set('Breaking-Bad', '<3');
    490 const headers = new Headers(meta);
    491 const copyOfHeaders = new Headers(headers);
    492 ```
    493 
    494 <a id="iface-body"></a>
    495 ### Interface: Body
    496 
    497 `Body` is an abstract interface with methods that are applicable to both `Request` and `Response` classes.
    498 
    499 The following methods are not yet implemented in node-fetch at this moment:
    500 
    501 - `formData()`
    502 
    503 #### body.body
    504 
    505 <small>*(deviation from spec)*</small>
    506 
    507 * Node.js [`Readable` stream][node-readable]
    508 
    509 Data are encapsulated in the `Body` object. Note that while the [Fetch Standard][whatwg-fetch] requires the property to always be a WHATWG `ReadableStream`, in node-fetch it is a Node.js [`Readable` stream][node-readable].
    510 
    511 #### body.bodyUsed
    512 
    513 <small>*(spec-compliant)*</small>
    514 
    515 * `Boolean`
    516 
    517 A boolean property for if this body has been consumed. Per the specs, a consumed body cannot be used again.
    518 
    519 #### body.arrayBuffer()
    520 #### body.blob()
    521 #### body.json()
    522 #### body.text()
    523 
    524 <small>*(spec-compliant)*</small>
    525 
    526 * Returns: <code>Promise</code>
    527 
    528 Consume the body and return a promise that will resolve to one of these formats.
    529 
    530 #### body.buffer()
    531 
    532 <small>*(node-fetch extension)*</small>
    533 
    534 * Returns: <code>Promise&lt;Buffer&gt;</code>
    535 
    536 Consume the body and return a promise that will resolve to a Buffer.
    537 
    538 #### body.textConverted()
    539 
    540 <small>*(node-fetch extension)*</small>
    541 
    542 * Returns: <code>Promise&lt;String&gt;</code>
    543 
    544 Identical to `body.text()`, except instead of always converting to UTF-8, encoding sniffing will be performed and text converted to UTF-8 if possible.
    545 
    546 (This API requires an optional dependency of the npm package [encoding](https://www.npmjs.com/package/encoding), which you need to install manually. `webpack` users may see [a warning message](https://github.com/bitinn/node-fetch/issues/412#issuecomment-379007792) due to this optional dependency.)
    547 
    548 <a id="class-fetcherror"></a>
    549 ### Class: FetchError
    550 
    551 <small>*(node-fetch extension)*</small>
    552 
    553 An operational error in the fetching process. See [ERROR-HANDLING.md][] for more info.
    554 
    555 <a id="class-aborterror"></a>
    556 ### Class: AbortError
    557 
    558 <small>*(node-fetch extension)*</small>
    559 
    560 An Error thrown when the request is aborted in response to an `AbortSignal`'s `abort` event. It has a `name` property of `AbortError`. See [ERROR-HANDLING.MD][] for more info.
    561 
    562 ## Acknowledgement
    563 
    564 Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid implementation reference.
    565 
    566 `node-fetch` v1 was maintained by [@bitinn](https://github.com/bitinn); v2 was maintained by [@TimothyGu](https://github.com/timothygu), [@bitinn](https://github.com/bitinn) and [@jimmywarting](https://github.com/jimmywarting); v2 readme is written by [@jkantr](https://github.com/jkantr).
    567 
    568 ## License
    569 
    570 MIT
    571 
    572 [npm-image]: https://flat.badgen.net/npm/v/node-fetch
    573 [npm-url]: https://www.npmjs.com/package/node-fetch
    574 [travis-image]: https://flat.badgen.net/travis/bitinn/node-fetch
    575 [travis-url]: https://travis-ci.org/bitinn/node-fetch
    576 [codecov-image]: https://flat.badgen.net/codecov/c/github/bitinn/node-fetch/master
    577 [codecov-url]: https://codecov.io/gh/bitinn/node-fetch
    578 [install-size-image]: https://flat.badgen.net/packagephobia/install/node-fetch
    579 [install-size-url]: https://packagephobia.now.sh/result?p=node-fetch
    580 [discord-image]: https://img.shields.io/discord/619915844268326952?color=%237289DA&label=Discord&style=flat-square
    581 [discord-url]: https://discord.gg/Zxbndcm
    582 [opencollective-image]: https://opencollective.com/node-fetch/backers.svg
    583 [opencollective-url]: https://opencollective.com/node-fetch
    584 [whatwg-fetch]: https://fetch.spec.whatwg.org/
    585 [response-init]: https://fetch.spec.whatwg.org/#responseinit
    586 [node-readable]: https://nodejs.org/api/stream.html#stream_readable_streams
    587 [mdn-headers]: https://developer.mozilla.org/en-US/docs/Web/API/Headers
    588 [LIMITS.md]: https://github.com/bitinn/node-fetch/blob/master/LIMITS.md
    589 [ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md
    590 [UPGRADE-GUIDE.md]: https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md