README.md (19782B)
1 node-fetch 2 ========== 3 4 [![npm version][npm-image]][npm-url] 5 [![build status][travis-image]][travis-url] 6 [![coverage status][codecov-image]][codecov-url] 7 [![install size][install-size-image]][install-size-url] 8 [![Discord][discord-image]][discord-url] 9 10 A light-weight module that brings `window.fetch` to Node.js 11 12 (We are looking for [v2 maintainers and collaborators](https://github.com/bitinn/node-fetch/issues/567)) 13 14 [![Backers][opencollective-image]][opencollective-url] 15 16 <!-- TOC --> 17 18 - [Motivation](#motivation) 19 - [Features](#features) 20 - [Difference from client-side fetch](#difference-from-client-side-fetch) 21 - [Installation](#installation) 22 - [Loading and configuring the module](#loading-and-configuring-the-module) 23 - [Common Usage](#common-usage) 24 - [Plain text or HTML](#plain-text-or-html) 25 - [JSON](#json) 26 - [Simple Post](#simple-post) 27 - [Post with JSON](#post-with-json) 28 - [Post with form parameters](#post-with-form-parameters) 29 - [Handling exceptions](#handling-exceptions) 30 - [Handling client and server errors](#handling-client-and-server-errors) 31 - [Advanced Usage](#advanced-usage) 32 - [Streams](#streams) 33 - [Buffer](#buffer) 34 - [Accessing Headers and other Meta data](#accessing-headers-and-other-meta-data) 35 - [Extract Set-Cookie Header](#extract-set-cookie-header) 36 - [Post data using a file stream](#post-data-using-a-file-stream) 37 - [Post with form-data (detect multipart)](#post-with-form-data-detect-multipart) 38 - [Request cancellation with AbortSignal](#request-cancellation-with-abortsignal) 39 - [API](#api) 40 - [fetch(url[, options])](#fetchurl-options) 41 - [Options](#options) 42 - [Class: Request](#class-request) 43 - [Class: Response](#class-response) 44 - [Class: Headers](#class-headers) 45 - [Interface: Body](#interface-body) 46 - [Class: FetchError](#class-fetcherror) 47 - [License](#license) 48 - [Acknowledgement](#acknowledgement) 49 50 <!-- /TOC --> 51 52 ## Motivation 53 54 Instead of implementing `XMLHttpRequest` in Node.js to run browser-specific [Fetch polyfill](https://github.com/github/fetch), why not go from native `http` to `fetch` API directly? Hence, `node-fetch`, minimal code for a `window.fetch` compatible API on Node.js runtime. 55 56 See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorphic-fetch) or Leonardo Quixada's [cross-fetch](https://github.com/lquixada/cross-fetch) for isomorphic usage (exports `node-fetch` for server-side, `whatwg-fetch` for client-side). 57 58 ## Features 59 60 - Stay consistent with `window.fetch` API. 61 - Make conscious trade-off when following [WHATWG fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known differences. 62 - Use native promise but allow substituting it with [insert your favorite promise library]. 63 - Use native Node streams for body on both request and response. 64 - Decode content encoding (gzip/deflate) properly and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically. 65 - Useful extensions such as timeout, redirect limit, response size limit, [explicit errors](ERROR-HANDLING.md) for troubleshooting. 66 67 ## Difference from client-side fetch 68 69 - See [Known Differences](LIMITS.md) for details. 70 - If you happen to use a missing feature that `window.fetch` offers, feel free to open an issue. 71 - Pull requests are welcomed too! 72 73 ## Installation 74 75 Current stable release (`2.x`) 76 77 ```sh 78 $ npm install node-fetch 79 ``` 80 81 ## Loading and configuring the module 82 We suggest you load the module via `require` until the stabilization of ES modules in node: 83 ```js 84 const fetch = require('node-fetch'); 85 ``` 86 87 If you are using a Promise library other than native, set it through `fetch.Promise`: 88 ```js 89 const Bluebird = require('bluebird'); 90 91 fetch.Promise = Bluebird; 92 ``` 93 94 ## Common Usage 95 96 NOTE: The documentation below is up-to-date with `2.x` releases; see the [`1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide](UPGRADE-GUIDE.md) for the differences. 97 98 #### Plain text or HTML 99 ```js 100 fetch('https://github.com/') 101 .then(res => res.text()) 102 .then(body => console.log(body)); 103 ``` 104 105 #### JSON 106 107 ```js 108 109 fetch('https://api.github.com/users/github') 110 .then(res => res.json()) 111 .then(json => console.log(json)); 112 ``` 113 114 #### Simple Post 115 ```js 116 fetch('https://httpbin.org/post', { method: 'POST', body: 'a=1' }) 117 .then(res => res.json()) // expecting a json response 118 .then(json => console.log(json)); 119 ``` 120 121 #### Post with JSON 122 123 ```js 124 const body = { a: 1 }; 125 126 fetch('https://httpbin.org/post', { 127 method: 'post', 128 body: JSON.stringify(body), 129 headers: { 'Content-Type': 'application/json' }, 130 }) 131 .then(res => res.json()) 132 .then(json => console.log(json)); 133 ``` 134 135 #### Post with form parameters 136 `URLSearchParams` is available in Node.js as of v7.5.0. See [official documentation](https://nodejs.org/api/url.html#url_class_urlsearchparams) for more usage methods. 137 138 NOTE: The `Content-Type` header is only set automatically to `x-www-form-urlencoded` when an instance of `URLSearchParams` is given as such: 139 140 ```js 141 const { URLSearchParams } = require('url'); 142 143 const params = new URLSearchParams(); 144 params.append('a', 1); 145 146 fetch('https://httpbin.org/post', { method: 'POST', body: params }) 147 .then(res => res.json()) 148 .then(json => console.log(json)); 149 ``` 150 151 #### Handling exceptions 152 NOTE: 3xx-5xx responses are *NOT* exceptions and should be handled in `then()`; see the next section for more information. 153 154 Adding a catch to the fetch promise chain will catch *all* exceptions, such as errors originating from node core libraries, network errors and operational errors, which are instances of FetchError. See the [error handling document](ERROR-HANDLING.md) for more details. 155 156 ```js 157 fetch('https://domain.invalid/') 158 .catch(err => console.error(err)); 159 ``` 160 161 #### Handling client and server errors 162 It is common to create a helper function to check that the response contains no client (4xx) or server (5xx) error responses: 163 164 ```js 165 function checkStatus(res) { 166 if (res.ok) { // res.status >= 200 && res.status < 300 167 return res; 168 } else { 169 throw MyCustomError(res.statusText); 170 } 171 } 172 173 fetch('https://httpbin.org/status/400') 174 .then(checkStatus) 175 .then(res => console.log('will not get here...')) 176 ``` 177 178 ## Advanced Usage 179 180 #### Streams 181 The "Node.js way" is to use streams when possible: 182 183 ```js 184 fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') 185 .then(res => { 186 const dest = fs.createWriteStream('./octocat.png'); 187 res.body.pipe(dest); 188 }); 189 ``` 190 191 #### Buffer 192 If you prefer to cache binary data in full, use buffer(). (NOTE: `buffer()` is a `node-fetch`-only API) 193 194 ```js 195 const fileType = require('file-type'); 196 197 fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') 198 .then(res => res.buffer()) 199 .then(buffer => fileType(buffer)) 200 .then(type => { /* ... */ }); 201 ``` 202 203 #### Accessing Headers and other Meta data 204 ```js 205 fetch('https://github.com/') 206 .then(res => { 207 console.log(res.ok); 208 console.log(res.status); 209 console.log(res.statusText); 210 console.log(res.headers.raw()); 211 console.log(res.headers.get('content-type')); 212 }); 213 ``` 214 215 #### Extract Set-Cookie Header 216 217 Unlike browsers, you can access raw `Set-Cookie` headers manually using `Headers.raw()`. This is a `node-fetch` only API. 218 219 ```js 220 fetch(url).then(res => { 221 // returns an array of values, instead of a string of comma-separated values 222 console.log(res.headers.raw()['set-cookie']); 223 }); 224 ``` 225 226 #### Post data using a file stream 227 228 ```js 229 const { createReadStream } = require('fs'); 230 231 const stream = createReadStream('input.txt'); 232 233 fetch('https://httpbin.org/post', { method: 'POST', body: stream }) 234 .then(res => res.json()) 235 .then(json => console.log(json)); 236 ``` 237 238 #### Post with form-data (detect multipart) 239 240 ```js 241 const FormData = require('form-data'); 242 243 const form = new FormData(); 244 form.append('a', 1); 245 246 fetch('https://httpbin.org/post', { method: 'POST', body: form }) 247 .then(res => res.json()) 248 .then(json => console.log(json)); 249 250 // OR, using custom headers 251 // NOTE: getHeaders() is non-standard API 252 253 const form = new FormData(); 254 form.append('a', 1); 255 256 const options = { 257 method: 'POST', 258 body: form, 259 headers: form.getHeaders() 260 } 261 262 fetch('https://httpbin.org/post', options) 263 .then(res => res.json()) 264 .then(json => console.log(json)); 265 ``` 266 267 #### Request cancellation with AbortSignal 268 269 > NOTE: You may cancel streamed requests only on Node >= v8.0.0 270 271 You may cancel requests with `AbortController`. A suggested implementation is [`abort-controller`](https://www.npmjs.com/package/abort-controller). 272 273 An example of timing out a request after 150ms could be achieved as the following: 274 275 ```js 276 import AbortController from 'abort-controller'; 277 278 const controller = new AbortController(); 279 const timeout = setTimeout( 280 () => { controller.abort(); }, 281 150, 282 ); 283 284 fetch(url, { signal: controller.signal }) 285 .then(res => res.json()) 286 .then( 287 data => { 288 useData(data) 289 }, 290 err => { 291 if (err.name === 'AbortError') { 292 // request was aborted 293 } 294 }, 295 ) 296 .finally(() => { 297 clearTimeout(timeout); 298 }); 299 ``` 300 301 See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for more examples. 302 303 304 ## API 305 306 ### fetch(url[, options]) 307 308 - `url` A string representing the URL for fetching 309 - `options` [Options](#fetch-options) for the HTTP(S) request 310 - Returns: <code>Promise<[Response](#class-response)></code> 311 312 Perform an HTTP(S) fetch. 313 314 `url` should be an absolute url, such as `https://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected `Promise`. 315 316 <a id="fetch-options"></a> 317 ### Options 318 319 The default values are shown after each option key. 320 321 ```js 322 { 323 // These properties are part of the Fetch Standard 324 method: 'GET', 325 headers: {}, // request headers. format is the identical to that accepted by the Headers constructor (see below) 326 body: null, // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream 327 redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect 328 signal: null, // pass an instance of AbortSignal to optionally abort requests 329 330 // The following properties are node-fetch extensions 331 follow: 20, // maximum redirect count. 0 to not follow redirect 332 timeout: 0, // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies). Signal is recommended instead. 333 compress: true, // support gzip/deflate content encoding. false to disable 334 size: 0, // maximum response body size in bytes. 0 to disable 335 agent: null // http(s).Agent instance or function that returns an instance (see below) 336 } 337 ``` 338 339 ##### Default Headers 340 341 If no values are set, the following request headers will be sent automatically: 342 343 Header | Value 344 ------------------- | -------------------------------------------------------- 345 `Accept-Encoding` | `gzip,deflate` _(when `options.compress === true`)_ 346 `Accept` | `*/*` 347 `Connection` | `close` _(when no `options.agent` is present)_ 348 `Content-Length` | _(automatically calculated, if possible)_ 349 `Transfer-Encoding` | `chunked` _(when `req.body` is a stream)_ 350 `User-Agent` | `node-fetch/1.0 (+https://github.com/bitinn/node-fetch)` 351 352 Note: when `body` is a `Stream`, `Content-Length` is not set automatically. 353 354 ##### Custom Agent 355 356 The `agent` option allows you to specify networking related options which are out of the scope of Fetch, including and not limited to the following: 357 358 - Support self-signed certificate 359 - Use only IPv4 or IPv6 360 - Custom DNS Lookup 361 362 See [`http.Agent`](https://nodejs.org/api/http.html#http_new_agent_options) for more information. 363 364 In addition, the `agent` option accepts a function that returns `http`(s)`.Agent` instance given current [URL](https://nodejs.org/api/url.html), this is useful during a redirection chain across HTTP and HTTPS protocol. 365 366 ```js 367 const httpAgent = new http.Agent({ 368 keepAlive: true 369 }); 370 const httpsAgent = new https.Agent({ 371 keepAlive: true 372 }); 373 374 const options = { 375 agent: function (_parsedURL) { 376 if (_parsedURL.protocol == 'http:') { 377 return httpAgent; 378 } else { 379 return httpsAgent; 380 } 381 } 382 } 383 ``` 384 385 <a id="class-request"></a> 386 ### Class: Request 387 388 An HTTP(S) request containing information about URL, method, headers, and the body. This class implements the [Body](#iface-body) interface. 389 390 Due to the nature of Node.js, the following properties are not implemented at this moment: 391 392 - `type` 393 - `destination` 394 - `referrer` 395 - `referrerPolicy` 396 - `mode` 397 - `credentials` 398 - `cache` 399 - `integrity` 400 - `keepalive` 401 402 The following node-fetch extension properties are provided: 403 404 - `follow` 405 - `compress` 406 - `counter` 407 - `agent` 408 409 See [options](#fetch-options) for exact meaning of these extensions. 410 411 #### new Request(input[, options]) 412 413 <small>*(spec-compliant)*</small> 414 415 - `input` A string representing a URL, or another `Request` (which will be cloned) 416 - `options` [Options][#fetch-options] for the HTTP(S) request 417 418 Constructs a new `Request` object. The constructor is identical to that in the [browser](https://developer.mozilla.org/en-US/docs/Web/API/Request/Request). 419 420 In most cases, directly `fetch(url, options)` is simpler than creating a `Request` object. 421 422 <a id="class-response"></a> 423 ### Class: Response 424 425 An HTTP(S) response. This class implements the [Body](#iface-body) interface. 426 427 The following properties are not implemented in node-fetch at this moment: 428 429 - `Response.error()` 430 - `Response.redirect()` 431 - `type` 432 - `trailer` 433 434 #### new Response([body[, options]]) 435 436 <small>*(spec-compliant)*</small> 437 438 - `body` A `String` or [`Readable` stream][node-readable] 439 - `options` A [`ResponseInit`][response-init] options dictionary 440 441 Constructs a new `Response` object. The constructor is identical to that in the [browser](https://developer.mozilla.org/en-US/docs/Web/API/Response/Response). 442 443 Because Node.js does not implement service workers (for which this class was designed), one rarely has to construct a `Response` directly. 444 445 #### response.ok 446 447 <small>*(spec-compliant)*</small> 448 449 Convenience property representing if the request ended normally. Will evaluate to true if the response status was greater than or equal to 200 but smaller than 300. 450 451 #### response.redirected 452 453 <small>*(spec-compliant)*</small> 454 455 Convenience property representing if the request has been redirected at least once. Will evaluate to true if the internal redirect counter is greater than 0. 456 457 <a id="class-headers"></a> 458 ### Class: Headers 459 460 This class allows manipulating and iterating over a set of HTTP headers. All methods specified in the [Fetch Standard][whatwg-fetch] are implemented. 461 462 #### new Headers([init]) 463 464 <small>*(spec-compliant)*</small> 465 466 - `init` Optional argument to pre-fill the `Headers` object 467 468 Construct a new `Headers` object. `init` can be either `null`, a `Headers` object, an key-value map object or any iterable object. 469 470 ```js 471 // Example adapted from https://fetch.spec.whatwg.org/#example-headers-class 472 473 const meta = { 474 'Content-Type': 'text/xml', 475 'Breaking-Bad': '<3' 476 }; 477 const headers = new Headers(meta); 478 479 // The above is equivalent to 480 const meta = [ 481 [ 'Content-Type', 'text/xml' ], 482 [ 'Breaking-Bad', '<3' ] 483 ]; 484 const headers = new Headers(meta); 485 486 // You can in fact use any iterable objects, like a Map or even another Headers 487 const meta = new Map(); 488 meta.set('Content-Type', 'text/xml'); 489 meta.set('Breaking-Bad', '<3'); 490 const headers = new Headers(meta); 491 const copyOfHeaders = new Headers(headers); 492 ``` 493 494 <a id="iface-body"></a> 495 ### Interface: Body 496 497 `Body` is an abstract interface with methods that are applicable to both `Request` and `Response` classes. 498 499 The following methods are not yet implemented in node-fetch at this moment: 500 501 - `formData()` 502 503 #### body.body 504 505 <small>*(deviation from spec)*</small> 506 507 * Node.js [`Readable` stream][node-readable] 508 509 Data are encapsulated in the `Body` object. Note that while the [Fetch Standard][whatwg-fetch] requires the property to always be a WHATWG `ReadableStream`, in node-fetch it is a Node.js [`Readable` stream][node-readable]. 510 511 #### body.bodyUsed 512 513 <small>*(spec-compliant)*</small> 514 515 * `Boolean` 516 517 A boolean property for if this body has been consumed. Per the specs, a consumed body cannot be used again. 518 519 #### body.arrayBuffer() 520 #### body.blob() 521 #### body.json() 522 #### body.text() 523 524 <small>*(spec-compliant)*</small> 525 526 * Returns: <code>Promise</code> 527 528 Consume the body and return a promise that will resolve to one of these formats. 529 530 #### body.buffer() 531 532 <small>*(node-fetch extension)*</small> 533 534 * Returns: <code>Promise<Buffer></code> 535 536 Consume the body and return a promise that will resolve to a Buffer. 537 538 #### body.textConverted() 539 540 <small>*(node-fetch extension)*</small> 541 542 * Returns: <code>Promise<String></code> 543 544 Identical to `body.text()`, except instead of always converting to UTF-8, encoding sniffing will be performed and text converted to UTF-8 if possible. 545 546 (This API requires an optional dependency of the npm package [encoding](https://www.npmjs.com/package/encoding), which you need to install manually. `webpack` users may see [a warning message](https://github.com/bitinn/node-fetch/issues/412#issuecomment-379007792) due to this optional dependency.) 547 548 <a id="class-fetcherror"></a> 549 ### Class: FetchError 550 551 <small>*(node-fetch extension)*</small> 552 553 An operational error in the fetching process. See [ERROR-HANDLING.md][] for more info. 554 555 <a id="class-aborterror"></a> 556 ### Class: AbortError 557 558 <small>*(node-fetch extension)*</small> 559 560 An Error thrown when the request is aborted in response to an `AbortSignal`'s `abort` event. It has a `name` property of `AbortError`. See [ERROR-HANDLING.MD][] for more info. 561 562 ## Acknowledgement 563 564 Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid implementation reference. 565 566 `node-fetch` v1 was maintained by [@bitinn](https://github.com/bitinn); v2 was maintained by [@TimothyGu](https://github.com/timothygu), [@bitinn](https://github.com/bitinn) and [@jimmywarting](https://github.com/jimmywarting); v2 readme is written by [@jkantr](https://github.com/jkantr). 567 568 ## License 569 570 MIT 571 572 [npm-image]: https://flat.badgen.net/npm/v/node-fetch 573 [npm-url]: https://www.npmjs.com/package/node-fetch 574 [travis-image]: https://flat.badgen.net/travis/bitinn/node-fetch 575 [travis-url]: https://travis-ci.org/bitinn/node-fetch 576 [codecov-image]: https://flat.badgen.net/codecov/c/github/bitinn/node-fetch/master 577 [codecov-url]: https://codecov.io/gh/bitinn/node-fetch 578 [install-size-image]: https://flat.badgen.net/packagephobia/install/node-fetch 579 [install-size-url]: https://packagephobia.now.sh/result?p=node-fetch 580 [discord-image]: https://img.shields.io/discord/619915844268326952?color=%237289DA&label=Discord&style=flat-square 581 [discord-url]: https://discord.gg/Zxbndcm 582 [opencollective-image]: https://opencollective.com/node-fetch/backers.svg 583 [opencollective-url]: https://opencollective.com/node-fetch 584 [whatwg-fetch]: https://fetch.spec.whatwg.org/ 585 [response-init]: https://fetch.spec.whatwg.org/#responseinit 586 [node-readable]: https://nodejs.org/api/stream.html#stream_readable_streams 587 [mdn-headers]: https://developer.mozilla.org/en-US/docs/Web/API/Headers 588 [LIMITS.md]: https://github.com/bitinn/node-fetch/blob/master/LIMITS.md 589 [ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md 590 [UPGRADE-GUIDE.md]: https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md