README.md (19335B)
1 node-fetch 2 ========== 3 4 [![npm version][npm-image]][npm-url] 5 [![build status][travis-image]][travis-url] 6 [![coverage status][codecov-image]][codecov-url] 7 [![install size][install-size-image]][install-size-url] 8 9 A light-weight module that brings `window.fetch` to Node.js 10 11 (We are looking for [v2 maintainers and collaborators](https://github.com/bitinn/node-fetch/issues/567)) 12 13 <!-- TOC --> 14 15 - [Motivation](#motivation) 16 - [Features](#features) 17 - [Difference from client-side fetch](#difference-from-client-side-fetch) 18 - [Installation](#installation) 19 - [Loading and configuring the module](#loading-and-configuring-the-module) 20 - [Common Usage](#common-usage) 21 - [Plain text or HTML](#plain-text-or-html) 22 - [JSON](#json) 23 - [Simple Post](#simple-post) 24 - [Post with JSON](#post-with-json) 25 - [Post with form parameters](#post-with-form-parameters) 26 - [Handling exceptions](#handling-exceptions) 27 - [Handling client and server errors](#handling-client-and-server-errors) 28 - [Advanced Usage](#advanced-usage) 29 - [Streams](#streams) 30 - [Buffer](#buffer) 31 - [Accessing Headers and other Meta data](#accessing-headers-and-other-meta-data) 32 - [Extract Set-Cookie Header](#extract-set-cookie-header) 33 - [Post data using a file stream](#post-data-using-a-file-stream) 34 - [Post with form-data (detect multipart)](#post-with-form-data-detect-multipart) 35 - [Request cancellation with AbortSignal](#request-cancellation-with-abortsignal) 36 - [API](#api) 37 - [fetch(url[, options])](#fetchurl-options) 38 - [Options](#options) 39 - [Class: Request](#class-request) 40 - [Class: Response](#class-response) 41 - [Class: Headers](#class-headers) 42 - [Interface: Body](#interface-body) 43 - [Class: FetchError](#class-fetcherror) 44 - [License](#license) 45 - [Acknowledgement](#acknowledgement) 46 47 <!-- /TOC --> 48 49 ## Motivation 50 51 Instead of implementing `XMLHttpRequest` in Node.js to run browser-specific [Fetch polyfill](https://github.com/github/fetch), why not go from native `http` to `fetch` API directly? Hence `node-fetch`, minimal code for a `window.fetch` compatible API on Node.js runtime. 52 53 See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorphic-fetch) or Leonardo Quixada's [cross-fetch](https://github.com/lquixada/cross-fetch) for isomorphic usage (exports `node-fetch` for server-side, `whatwg-fetch` for client-side). 54 55 ## Features 56 57 - Stay consistent with `window.fetch` API. 58 - Make conscious trade-off when following [WHATWG fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known differences. 59 - Use native promise, but allow substituting it with [insert your favorite promise library]. 60 - Use native Node streams for body, on both request and response. 61 - Decode content encoding (gzip/deflate) properly, and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically. 62 - Useful extensions such as timeout, redirect limit, response size limit, [explicit errors](ERROR-HANDLING.md) for troubleshooting. 63 64 ## Difference from client-side fetch 65 66 - See [Known Differences](LIMITS.md) for details. 67 - If you happen to use a missing feature that `window.fetch` offers, feel free to open an issue. 68 - Pull requests are welcomed too! 69 70 ## Installation 71 72 Current stable release (`2.x`) 73 74 ```sh 75 $ npm install node-fetch --save 76 ``` 77 78 ## Loading and configuring the module 79 We suggest you load the module via `require`, pending the stabalizing of es modules in node: 80 ```js 81 const fetch = require('node-fetch'); 82 ``` 83 84 If you are using a Promise library other than native, set it through fetch.Promise: 85 ```js 86 const Bluebird = require('bluebird'); 87 88 fetch.Promise = Bluebird; 89 ``` 90 91 ## Common Usage 92 93 NOTE: The documentation below is up-to-date with `2.x` releases, [see `1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide](UPGRADE-GUIDE.md) for the differences. 94 95 #### Plain text or HTML 96 ```js 97 fetch('https://github.com/') 98 .then(res => res.text()) 99 .then(body => console.log(body)); 100 ``` 101 102 #### JSON 103 104 ```js 105 106 fetch('https://api.github.com/users/github') 107 .then(res => res.json()) 108 .then(json => console.log(json)); 109 ``` 110 111 #### Simple Post 112 ```js 113 fetch('https://httpbin.org/post', { method: 'POST', body: 'a=1' }) 114 .then(res => res.json()) // expecting a json response 115 .then(json => console.log(json)); 116 ``` 117 118 #### Post with JSON 119 120 ```js 121 const body = { a: 1 }; 122 123 fetch('https://httpbin.org/post', { 124 method: 'post', 125 body: JSON.stringify(body), 126 headers: { 'Content-Type': 'application/json' }, 127 }) 128 .then(res => res.json()) 129 .then(json => console.log(json)); 130 ``` 131 132 #### Post with form parameters 133 `URLSearchParams` is available in Node.js as of v7.5.0. See [official documentation](https://nodejs.org/api/url.html#url_class_urlsearchparams) for more usage methods. 134 135 NOTE: The `Content-Type` header is only set automatically to `x-www-form-urlencoded` when an instance of `URLSearchParams` is given as such: 136 137 ```js 138 const { URLSearchParams } = require('url'); 139 140 const params = new URLSearchParams(); 141 params.append('a', 1); 142 143 fetch('https://httpbin.org/post', { method: 'POST', body: params }) 144 .then(res => res.json()) 145 .then(json => console.log(json)); 146 ``` 147 148 #### Handling exceptions 149 NOTE: 3xx-5xx responses are *NOT* exceptions, and should be handled in `then()`, see the next section. 150 151 Adding a catch to the fetch promise chain will catch *all* exceptions, such as errors originating from node core libraries, like network errors, and operational errors which are instances of FetchError. See the [error handling document](ERROR-HANDLING.md) for more details. 152 153 ```js 154 fetch('https://domain.invalid/') 155 .catch(err => console.error(err)); 156 ``` 157 158 #### Handling client and server errors 159 It is common to create a helper function to check that the response contains no client (4xx) or server (5xx) error responses: 160 161 ```js 162 function checkStatus(res) { 163 if (res.ok) { // res.status >= 200 && res.status < 300 164 return res; 165 } else { 166 throw MyCustomError(res.statusText); 167 } 168 } 169 170 fetch('https://httpbin.org/status/400') 171 .then(checkStatus) 172 .then(res => console.log('will not get here...')) 173 ``` 174 175 ## Advanced Usage 176 177 #### Streams 178 The "Node.js way" is to use streams when possible: 179 180 ```js 181 fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') 182 .then(res => { 183 const dest = fs.createWriteStream('./octocat.png'); 184 res.body.pipe(dest); 185 }); 186 ``` 187 188 #### Buffer 189 If you prefer to cache binary data in full, use buffer(). (NOTE: buffer() is a `node-fetch` only API) 190 191 ```js 192 const fileType = require('file-type'); 193 194 fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') 195 .then(res => res.buffer()) 196 .then(buffer => fileType(buffer)) 197 .then(type => { /* ... */ }); 198 ``` 199 200 #### Accessing Headers and other Meta data 201 ```js 202 fetch('https://github.com/') 203 .then(res => { 204 console.log(res.ok); 205 console.log(res.status); 206 console.log(res.statusText); 207 console.log(res.headers.raw()); 208 console.log(res.headers.get('content-type')); 209 }); 210 ``` 211 212 #### Extract Set-Cookie Header 213 214 Unlike browsers, you can access raw `Set-Cookie` headers manually using `Headers.raw()`, this is a `node-fetch` only API. 215 216 ```js 217 fetch(url).then(res => { 218 // returns an array of values, instead of a string of comma-separated values 219 console.log(res.headers.raw()['set-cookie']); 220 }); 221 ``` 222 223 #### Post data using a file stream 224 225 ```js 226 const { createReadStream } = require('fs'); 227 228 const stream = createReadStream('input.txt'); 229 230 fetch('https://httpbin.org/post', { method: 'POST', body: stream }) 231 .then(res => res.json()) 232 .then(json => console.log(json)); 233 ``` 234 235 #### Post with form-data (detect multipart) 236 237 ```js 238 const FormData = require('form-data'); 239 240 const form = new FormData(); 241 form.append('a', 1); 242 243 fetch('https://httpbin.org/post', { method: 'POST', body: form }) 244 .then(res => res.json()) 245 .then(json => console.log(json)); 246 247 // OR, using custom headers 248 // NOTE: getHeaders() is non-standard API 249 250 const form = new FormData(); 251 form.append('a', 1); 252 253 const options = { 254 method: 'POST', 255 body: form, 256 headers: form.getHeaders() 257 } 258 259 fetch('https://httpbin.org/post', options) 260 .then(res => res.json()) 261 .then(json => console.log(json)); 262 ``` 263 264 #### Request cancellation with AbortSignal 265 266 > NOTE: You may only cancel streamed requests on Node >= v8.0.0 267 268 You may cancel requests with `AbortController`. A suggested implementation is [`abort-controller`](https://www.npmjs.com/package/abort-controller). 269 270 An example of timing out a request after 150ms could be achieved as follows: 271 272 ```js 273 import AbortController from 'abort-controller'; 274 275 const controller = new AbortController(); 276 const timeout = setTimeout( 277 () => { controller.abort(); }, 278 150, 279 ); 280 281 fetch(url, { signal: controller.signal }) 282 .then(res => res.json()) 283 .then( 284 data => { 285 useData(data) 286 }, 287 err => { 288 if (err.name === 'AbortError') { 289 // request was aborted 290 } 291 }, 292 ) 293 .finally(() => { 294 clearTimeout(timeout); 295 }); 296 ``` 297 298 See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for more examples. 299 300 301 ## API 302 303 ### fetch(url[, options]) 304 305 - `url` A string representing the URL for fetching 306 - `options` [Options](#fetch-options) for the HTTP(S) request 307 - Returns: <code>Promise<[Response](#class-response)></code> 308 309 Perform an HTTP(S) fetch. 310 311 `url` should be an absolute url, such as `https://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected promise. 312 313 <a id="fetch-options"></a> 314 ### Options 315 316 The default values are shown after each option key. 317 318 ```js 319 { 320 // These properties are part of the Fetch Standard 321 method: 'GET', 322 headers: {}, // request headers. format is the identical to that accepted by the Headers constructor (see below) 323 body: null, // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream 324 redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect 325 signal: null, // pass an instance of AbortSignal to optionally abort requests 326 327 // The following properties are node-fetch extensions 328 follow: 20, // maximum redirect count. 0 to not follow redirect 329 timeout: 0, // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies). Signal is recommended instead. 330 compress: true, // support gzip/deflate content encoding. false to disable 331 size: 0, // maximum response body size in bytes. 0 to disable 332 agent: null // http(s).Agent instance or function that returns an instance (see below) 333 } 334 ``` 335 336 ##### Default Headers 337 338 If no values are set, the following request headers will be sent automatically: 339 340 Header | Value 341 ------------------- | -------------------------------------------------------- 342 `Accept-Encoding` | `gzip,deflate` _(when `options.compress === true`)_ 343 `Accept` | `*/*` 344 `Connection` | `close` _(when no `options.agent` is present)_ 345 `Content-Length` | _(automatically calculated, if possible)_ 346 `Transfer-Encoding` | `chunked` _(when `req.body` is a stream)_ 347 `User-Agent` | `node-fetch/1.0 (+https://github.com/bitinn/node-fetch)` 348 349 Note: when `body` is a `Stream`, `Content-Length` is not set automatically. 350 351 ##### Custom Agent 352 353 The `agent` option allows you to specify networking related options that's out of the scope of Fetch. Including and not limit to: 354 355 - Support self-signed certificate 356 - Use only IPv4 or IPv6 357 - Custom DNS Lookup 358 359 See [`http.Agent`](https://nodejs.org/api/http.html#http_new_agent_options) for more information. 360 361 In addition, `agent` option accepts a function that returns http(s).Agent instance given current [URL](https://nodejs.org/api/url.html), this is useful during a redirection chain across HTTP and HTTPS protocol. 362 363 ```js 364 const httpAgent = new http.Agent({ 365 keepAlive: true 366 }); 367 const httpsAgent = new https.Agent({ 368 keepAlive: true 369 }); 370 371 const options = { 372 agent: function (_parsedURL) { 373 if (_parsedURL.protocol == 'http:') { 374 return httpAgent; 375 } else { 376 return httpsAgent; 377 } 378 } 379 } 380 ``` 381 382 <a id="class-request"></a> 383 ### Class: Request 384 385 An HTTP(S) request containing information about URL, method, headers, and the body. This class implements the [Body](#iface-body) interface. 386 387 Due to the nature of Node.js, the following properties are not implemented at this moment: 388 389 - `type` 390 - `destination` 391 - `referrer` 392 - `referrerPolicy` 393 - `mode` 394 - `credentials` 395 - `cache` 396 - `integrity` 397 - `keepalive` 398 399 The following node-fetch extension properties are provided: 400 401 - `follow` 402 - `compress` 403 - `counter` 404 - `agent` 405 406 See [options](#fetch-options) for exact meaning of these extensions. 407 408 #### new Request(input[, options]) 409 410 <small>*(spec-compliant)*</small> 411 412 - `input` A string representing a URL, or another `Request` (which will be cloned) 413 - `options` [Options][#fetch-options] for the HTTP(S) request 414 415 Constructs a new `Request` object. The constructor is identical to that in the [browser](https://developer.mozilla.org/en-US/docs/Web/API/Request/Request). 416 417 In most cases, directly `fetch(url, options)` is simpler than creating a `Request` object. 418 419 <a id="class-response"></a> 420 ### Class: Response 421 422 An HTTP(S) response. This class implements the [Body](#iface-body) interface. 423 424 The following properties are not implemented in node-fetch at this moment: 425 426 - `Response.error()` 427 - `Response.redirect()` 428 - `type` 429 - `trailer` 430 431 #### new Response([body[, options]]) 432 433 <small>*(spec-compliant)*</small> 434 435 - `body` A string or [Readable stream][node-readable] 436 - `options` A [`ResponseInit`][response-init] options dictionary 437 438 Constructs a new `Response` object. The constructor is identical to that in the [browser](https://developer.mozilla.org/en-US/docs/Web/API/Response/Response). 439 440 Because Node.js does not implement service workers (for which this class was designed), one rarely has to construct a `Response` directly. 441 442 #### response.ok 443 444 <small>*(spec-compliant)*</small> 445 446 Convenience property representing if the request ended normally. Will evaluate to true if the response status was greater than or equal to 200 but smaller than 300. 447 448 #### response.redirected 449 450 <small>*(spec-compliant)*</small> 451 452 Convenience property representing if the request has been redirected at least once. Will evaluate to true if the internal redirect counter is greater than 0. 453 454 <a id="class-headers"></a> 455 ### Class: Headers 456 457 This class allows manipulating and iterating over a set of HTTP headers. All methods specified in the [Fetch Standard][whatwg-fetch] are implemented. 458 459 #### new Headers([init]) 460 461 <small>*(spec-compliant)*</small> 462 463 - `init` Optional argument to pre-fill the `Headers` object 464 465 Construct a new `Headers` object. `init` can be either `null`, a `Headers` object, an key-value map object, or any iterable object. 466 467 ```js 468 // Example adapted from https://fetch.spec.whatwg.org/#example-headers-class 469 470 const meta = { 471 'Content-Type': 'text/xml', 472 'Breaking-Bad': '<3' 473 }; 474 const headers = new Headers(meta); 475 476 // The above is equivalent to 477 const meta = [ 478 [ 'Content-Type', 'text/xml' ], 479 [ 'Breaking-Bad', '<3' ] 480 ]; 481 const headers = new Headers(meta); 482 483 // You can in fact use any iterable objects, like a Map or even another Headers 484 const meta = new Map(); 485 meta.set('Content-Type', 'text/xml'); 486 meta.set('Breaking-Bad', '<3'); 487 const headers = new Headers(meta); 488 const copyOfHeaders = new Headers(headers); 489 ``` 490 491 <a id="iface-body"></a> 492 ### Interface: Body 493 494 `Body` is an abstract interface with methods that are applicable to both `Request` and `Response` classes. 495 496 The following methods are not yet implemented in node-fetch at this moment: 497 498 - `formData()` 499 500 #### body.body 501 502 <small>*(deviation from spec)*</small> 503 504 * Node.js [`Readable` stream][node-readable] 505 506 The data encapsulated in the `Body` object. Note that while the [Fetch Standard][whatwg-fetch] requires the property to always be a WHATWG `ReadableStream`, in node-fetch it is a Node.js [`Readable` stream][node-readable]. 507 508 #### body.bodyUsed 509 510 <small>*(spec-compliant)*</small> 511 512 * `Boolean` 513 514 A boolean property for if this body has been consumed. Per spec, a consumed body cannot be used again. 515 516 #### body.arrayBuffer() 517 #### body.blob() 518 #### body.json() 519 #### body.text() 520 521 <small>*(spec-compliant)*</small> 522 523 * Returns: <code>Promise</code> 524 525 Consume the body and return a promise that will resolve to one of these formats. 526 527 #### body.buffer() 528 529 <small>*(node-fetch extension)*</small> 530 531 * Returns: <code>Promise<Buffer></code> 532 533 Consume the body and return a promise that will resolve to a Buffer. 534 535 #### body.textConverted() 536 537 <small>*(node-fetch extension)*</small> 538 539 * Returns: <code>Promise<String></code> 540 541 Identical to `body.text()`, except instead of always converting to UTF-8, encoding sniffing will be performed and text converted to UTF-8, if possible. 542 543 (This API requires an optional dependency on npm package [encoding](https://www.npmjs.com/package/encoding), which you need to install manually. `webpack` users may see [a warning message](https://github.com/bitinn/node-fetch/issues/412#issuecomment-379007792) due to this optional dependency.) 544 545 <a id="class-fetcherror"></a> 546 ### Class: FetchError 547 548 <small>*(node-fetch extension)*</small> 549 550 An operational error in the fetching process. See [ERROR-HANDLING.md][] for more info. 551 552 <a id="class-aborterror"></a> 553 ### Class: AbortError 554 555 <small>*(node-fetch extension)*</small> 556 557 An Error thrown when the request is aborted in response to an `AbortSignal`'s `abort` event. It has a `name` property of `AbortError`. See [ERROR-HANDLING.MD][] for more info. 558 559 ## Acknowledgement 560 561 Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid implementation reference. 562 563 `node-fetch` v1 was maintained by [@bitinn](https://github.com/bitinn); v2 was maintained by [@TimothyGu](https://github.com/timothygu), [@bitinn](https://github.com/bitinn) and [@jimmywarting](https://github.com/jimmywarting); v2 readme is written by [@jkantr](https://github.com/jkantr). 564 565 ## License 566 567 MIT 568 569 [npm-image]: https://flat.badgen.net/npm/v/node-fetch 570 [npm-url]: https://www.npmjs.com/package/node-fetch 571 [travis-image]: https://flat.badgen.net/travis/bitinn/node-fetch 572 [travis-url]: https://travis-ci.org/bitinn/node-fetch 573 [codecov-image]: https://flat.badgen.net/codecov/c/github/bitinn/node-fetch/master 574 [codecov-url]: https://codecov.io/gh/bitinn/node-fetch 575 [install-size-image]: https://flat.badgen.net/packagephobia/install/node-fetch 576 [install-size-url]: https://packagephobia.now.sh/result?p=node-fetch 577 [whatwg-fetch]: https://fetch.spec.whatwg.org/ 578 [response-init]: https://fetch.spec.whatwg.org/#responseinit 579 [node-readable]: https://nodejs.org/api/stream.html#stream_readable_streams 580 [mdn-headers]: https://developer.mozilla.org/en-US/docs/Web/API/Headers 581 [LIMITS.md]: https://github.com/bitinn/node-fetch/blob/master/LIMITS.md 582 [ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md 583 [UPGRADE-GUIDE.md]: https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md